Nine years later, one feature in GTA4 has never been bettered - here's its story
A staggering piece of work.
Charting the phenomenal progress of video games these past few decades is easy enough. You've probably seen an image of PlayStation-era Lara Croft in all her stark polygonal beauty contrasted side-by-side with her modern character model. If not, perhaps you've seen Wolfenstein: The New Order's B.J. Blazkowicz sat next to his coarsely drawn early incarnation. Look at all those pixels, all that detail, and marvel at how far we've come.
Yet while this visual arms race continues, character physics are being left to stagnate. You might be able to bring down massive structures by taking out support beams on Red Faction: Guerilla's open-world Mars, and Microsoft may be promising to let us topple entire skyscrapers with the power of the cloud for Crackdown 3's multiplayer, but character models are left to flop to the floor, float above the ground, wibble through scenery, or keel over in a canned animation. Grand Theft Auto 4, though, was different.
Going on a rocket rampage in Liberty City is so cliché. Grand Theft Auto 4's greatest crimes happen when you channel your inner Agent 47 and push people to their deaths: over the edge of a bridge, onto the railway tracks, down some stairs - whatever works for you, you miserable sadist.
Rockstar knew exactly what it was doing when it added a button to allow players to shove NPCs. Playing the original Assassin's Creed, I used to use the 'gentle push' interaction to send its street-wandering drones to a watery death, but you could tell this wasn't its intended purpose. When you see the slapstick tumble of a GTA4 NPC falling violently down a flight of stairs, though, you know this is what the developers wanted. All that guilt ebbs away.
When the open-world crime 'em up released in 2008, it felt like the next step. Powered by Euphoria and working in synergy with Rockstar's in-house RAGE engine, characters would react to situations dynamically - grabbing at bullet wounds, tripping down staircases, stumbling while intoxicated, bouncing off objects, and reacting to their surroundings once prodded. It was a revelation, but outside of a few rare examples - Backbreaker and Star Wars: The Force Unleashed - it's rarely been seen outside of Rockstar's own games since.
Harry Denholm worked on GTA4 as engineering lead at NaturalMotion, the company behind Euphoria. When he started at the company in 2004, he was part of a six-person incubatory team that created the technology, taking inspiration from a piece of NaturalMotion's existing tech dubbed Endorphin and reinventing it so it could be used in real-time. The incubatory team would then work closely with game studios to plug it into their games.
"It was a few years in when the company entered into discussions with Rockstar about the potential to use this new approach to character animation in one of their upcoming titles, Red Dead: Redemption," Denholm explains. "A small team was assembled at NaturalMotion to start work on integrating with RAGE and iterating directly with gameplay and animation teams in San Diego. I was the technical lead of that team, handling the RAGE integration, runtime engineering and engine-specific tools work."
A couple of years later, Denholm created a showreel for the tech the team had been building for Red Dead: Redemption, but transported to a more contemporary action movie setting. The video saw characters being clipped by cars, dragged at gunpoint, thrown down stairs, pepper sprayed, and more, and it was all fully dynamic and interactive, with variable results every time. In a twist of fate, a schedule change saw the team migrate from Red Dead: Redemption over to GTA4, where they would create similar scenarios in Rockstar's urban sandbox.
Euphoria is special because it combines on-the-fly animation with AI, biomechanics, and physics, all without the need for motion capture. The results vary because the CPU is forcing the characters to react - flinching when something comes close, grasping at injuries, stumbling backwards when slightly knocked, and tripping if an object is placed in their way as they stumble. The characters have self-preservation hard-coded into them, so they can sometimes recover from a knock or get out of the way of an incoming threat, but it's never canned, making it unpredictable and exciting for the player. It's why it's so devilishly fun to fly a helicopter up to a Liberty City power plant and let the rotors have their way with the workmen up top, essentially.
It's now almost a decade after the release of GTA4, and we're still getting games that ape the series' open-world design, but they lack the small details that make the simulation exciting. If you've ever spent hours launching Trevor and co into a brick wall in GTA5, you will know what I mean: the satisfying slap, the way they crumple to the floor, and the different result every time makes it sadistically satisfying. Meanwhile, get hit by a car in Watch Dogs 2's San Francisco and you'll likely be unceremoniously bumped aside or end up stood on its roof. If a car hits you in Liberty City, there are countless ways it could play out: your head could bounce off the bonnet or windscreen, you could be thrown over the top, clipped aside, ran over. It feels like anything could happen.
You might be wondering, then, why other triple-A studios didn't pick up the tech straight after GTA4's release. "Euphoria was quite an unusual business proposition," explains Denholm. "It didn't exist fully-featured off-the-shelf in the way a physics engine does - what we were essentially selling was an office full of smart, dedicated people and the promise of a unique feature that no-one else would have.
"Our process was usually to visit an interested company - such as Rockstar or LucasArts - and an engineer would spend some time, often on-site, building a bespoke integration with their existing game code and physics engine. In the background, our behaviour engineers - programmers with backgrounds in physics, biomechanics, robotic control theory and/or AI - would begin sketching out basic behaviours on top of that integration. We would demo some project-specific behaviours and, if the deal got done, the engineers would shift to working solely on that in-step with the game's own schedule."
It was a bespoke, costly service. All Euphoria projects were built from scratch, taking on a laser-focused approach to suit the game they were integrating with. NaturalMotion found this delivered the best results. It's also why you can throw Euphoria powered ragdolls around with the Force in Star Wars: The Force Awakens, while the ones seen in GTA4 are more likely to grab at their car's door handle as you speed away in their recently-nabbed vehicle.
"The effort required was substantial and sustained," Denholm tells me. "Also, a Euphoria licence was a pretty expensive investment that involved long-term commitment from both sides, not a casual one-off upfront payment that just drops into your game. That model simply isn't a workable fit for a lot of developers."
These days, Euphoria is much easier to implement, being delivered along with NaturalMotion's Morpheme middleware. Morpheme has been used in Quantum Break, Until Dawn, and Killzone: Shadow Fall. However, none of these games' ragdolls and character physics match those in Rockstar's games, because they lack that expensive, tailor-made integration. Of course, it helped that Rockstar's RAGE engine and work ethic were a perfect fit.
"RAGE's physics ended up being a really potent platform for the Euphoria integration; we were able to pull off effects that the other commercial physics engines struggled with," remembers Denholm.
Rockstar was focused on creating a simulated world, from the AI routines of its NPCs the nuances in how they move, all ruled by lifelike physics and generated in real-time. To help achieve this, NaturalMotion was provided full access to the physics source code for RAGE, so the entire thing was a full-on collaboration.
RAGE was a capable physics simulation on its own, but Rockstar was also working on a specialist simulation feature called the 'Featherstone articulated body method', which allowed more accurate simulation of a biped. Bipedal ragdolls are assembled from a collection of rigid collision shapes, almost like an artist's pencil outline, capturing the human form with crude cylinders and boxy appendages - a blank slate, ready for the details to be added later. Joints are placed between each shape, allowing each part of the ragdoll to move independently, with elbow hinges, ankles, knees, and even flex points along the torso. Motors are added to these joints, mimicking human muscles.
No doubt you've seen physics go wrong before, with appendages getting trapped in terrain, characters stretching out of shape, or vibrating on the floor. That's because conventional physics engines treat each part of a ragdoll as a separate entity. These gruesome contortions are the result of the motorised joints trying ferociously to put it right again when things go awry.
"The difference with the Featherstone approach is that the whole skeleton is treated as an indivisible collection of objects," explains Denholm. "We're not updating the forearm, then the upper arm, then getting the joint to try and ensure those two parts are still stuck together - instead the whole biped is processed as a complex hierarchy of parts that are, in theory, impossible to break apart. All the joints are considered as part of the whole, rather than in isolation."
This behaviour worked well with the 'Dynamic Balancer', some tech created by Tom Lowe, lead behaviour engineer at NaturalMotion. The Dynamic Balancer allowed GTA4's characters to stand up and balance autonomously. "It had to take rapid, accurate stabilising steps to level the character's balance, as well as handle the rest of the skeleton being balanced on top of the legs, constantly compensating for gravity, centre-of-mass shifts and external impacts," says Denholm.
"We didn't request [Featherstone] from Rockstar, they had already been developing it," Tom Lowe tells me over email. "They employed a professor there who built it." You see, the problem with bipeds handled by in-game physics is that they walk on their weakest point - essentially a small box attached to the ankle joint - so they require joint strength, which Featherstone supplied. "Featherstone allowed better control of the character, joint strengths that matched what you set them to (not weaker), good friction and it avoided foot jitter," explains Lowe.
"The upper body didn't sag or collapse; the legs, kept stiff and would just naturally provide full-body support. Just like real legs!" adds Denholm. "Driving one to take a load-bearing step and not have the hips pull apart or pitch the torso over... this was a serious improvement. We could have our sims grab onto fast moving objects or take a high-speed car impact to the torso without all the body parts scattering like Lego. Plus, being able to work without the simulation instability meant our behaviour team could then focus on creating more realistic looking behaviours that required fewer hacks and cheats to work or look good.
"By the end of GTA4 we had characters that could be shot, grab and react to the wounds, stagger about over uneven streets while still firing their weapons back at the player - with realistic gun shot recoils being fed back into physics - all entirely dynamically simulated. And this was then being applied to every NPC in the whole game. It was a huge deal - and phenomenal responsibility - for us. By comparison, another project using Euphoria on a different physics engine was constantly fighting instability and had to use layers of additional - and sometimes unpredictable - cheat forces to achieve a level of control."
So results varied from project to project, it was more computationally expensive than traditional methods, and not every studio can just hire a pet scientist. Not only that, it's something that's difficult to sell to a customer, too. Painstaking recreations of real-world locations offer the promise of virtual tourism. Gameplay gimmicks provide a hook. 'Realistic ragdolls' isn't exactly something developers can slap on the box, so most just choose to not bother with the costly, time-hungry process. More traditional, tailor-made animations are also much more gameplay trailer-friendly, and it's difficult to establish a consistent style when a dynamic simulation wrestles control from canned animations in specific circumstances.
"I think in the triple-A arena it's harder for these things to advance at such a rapid pace," Denholm muses. "Graphics sells, so having it as a priority is an easy choice for research and investment. Complicated, unproven control systems or intricate AI can take much longer to build and prove out - and sometimes, if they do work really well, they will often fade into the background of a game experience, unnoticed or underappreciated. Putting believable, dynamic, reactive characters in a fun sandbox world took us years to get right - it requires the trust and belief from all involved that it will be worth the journey."
Denholm believes the next step is in making this arduous process much more simple and less expensive, much like how developers have easy access to high-end game engines such as Unity and Unreal 4. Euphoria could even be packaged as a generic solution, useable as a plug-in for popular game engines. Admittedly, you still wouldn't get the results seen in Rockstar's games, but it would be an improvement over the current standard.
"That's not to say that with accessible tools (like Unity) and a talented dev community you couldn't start to move fast towards those kind of results," explains Denholm. "I know NaturalMotion offer Unreal integration for their latest Morpheme+Euphoria middleware, and it wouldn't surprise me at all if there were more hobbyist procedural animation tools that spring up over the next few years. We have shown what is possible with enough time and effort. Having these things in the hands of indie/hobbyist devs where the rules and expectations are relaxed gives me hope that there are exciting new experiences to be had we haven't even thought of yet."
Indie studio Punches Bears is currently creating something in Unity that's based around physics-led animations. Some of its current projects show off bears dancing to Nas, a bear rodeo, chainsaw-wielding bears - there's definitely a theme here somewhere. Outside of the bears, though, all the projects are using character physics in an interesting new way. Here's a little taster of the work-in-progress results:
Elsewhere, in the triple-A space, Ubisoft has been quietly tinkering away on improvements to character physics, too. The studio's 'IK Rig' prototype shows a character rig that's capable of smooth, reactive movement over shifting terrain. Another Ubisoft tech demo from last year's GDC also shows advancement in physics-led character animations, with bio-mechanically correct weight transfers, smooth acceleration and deceleration, and believable stumbling looping between animations and physics when struck. All of this demo was also player-controlled.
The development of these animation tools could well see results similar to Euphoria become commonplace over the next few years. Maybe someday we'll even see a game surpass the phenomenal physics that underpinned Rockstar's GTA4. We just need more developers to look beyond the surface details. Perhaps, like GTA4's NPCs, they just need a little push.