Inside Killzone: Mercenary
Digital Foundry vs. Guerrilla Cambridge on the tech secrets of mobile's most advanced FPS.
Out now in Europe and days away from its US launch, Killzone: Mercenary for PlayStation Vita is undoubtedly one of the most technologically impressive mobile games on the market, and the first truly worthwhile first-person shooter for Sony's portable games machine. Remember that feeling when you first played Ridge Racer on PSP? It's the feeling that you're playing a game that has no right to look this good on mobile hardware; the sensation that you're getting a full-fat console game running in the palm of your hand. Uncharted: Golden Abyss captured that feeling, and now Killzone: Mercenary makes its case as the definitive article.
From a technological perspective, we simply had to find out more, so we spoke to Guerrilla Cambridge about the game's development, making a number of remarkable discoveries in the process - not least the fact that the Killzone 3 engine was actually built to run on multiple platforms, as technical director Matt Porter explains.
"Killzone: Mercenary is built on top of the Killzone 3 codebase and, at a structural level, much of the framework is identical," he tells us. "Out of the box, the engine supports multiple build targets - PlayStation 3 and PC - so extending to a third platform was relatively painless. Relatively. It's been a long and evolutionary process to get from first drop of the source to finished game.
"At the lower hardware-interfacing levels, rendering, audio, network and control-system layers have seen most revision, as would be expected. We've retained the physics model (Havok) but have had to migrate much of the animation system to a new framework - a Sony Computer Entertainment internal library which had seen major revision since KZ3 shipped."
With the bombshell dropped that Killzone 3 could conceivably be ported to PC (and presumably more readily adaptable for PlayStation 4), Porter goes on to explain how the engine was adapted to suit the multi-core set-up of the Vita. Sony's handheld has a quad-core ARM Cortex A9 processor, but one of those cores is reserved for the operating system and security, leaving three for game developers. Up against the six SPUs and the PPU of the PS3's Cell chip, that's a significant difference to address.
"Obviously a major difference is the absence of SPUs in the hardware, so parallelism models have shifted to utilise the three cores," Porter affirms. "We started with the PC framework as reference here: SPU job deployment is modelled as a synchronous process on that platform, with the actual task functionality identical to the PS3. That's obviously a lot slower, but provided the hook for a custom job manager on Vita. Over time, we've migrated most of the original SPU jobs onto the Vita cores and returned back to a fully asynchronous model."
"Killzone: Mercenary is built on top of the Killzone 3 codebase and, at a structural level, much of the framework is identical... the engine supports multiple build targets - PlayStation 3 and PC - so extending to a third platform was relatively painless."
Porter also describes how further opportunities to run the Killzone codebase over multiple cores were explored, and also says that model-skinning - a task carried out on SPU on PS3 - was actually ported to the Vita's graphics chip for Mercenary. Overall, he is keen to point out that the key systems that define the Killzone experience are virtually unchanged from their implementation on PlayStation 3.
"At a game level, again many of the core components of a Killzone game remain untouched and worked pretty much out-the-box on Vita," he says. "The major additions have been mechanics built upon the touchscreen - for example, the front-end interactions, melee combat and the hacking game, the vanguard mechanics and the backend economy system."
How the Killzone tech was downscaled for Vita
One of the defining elements of the Killzone experience is the AI, and Guerrilla Cambridge was keen to retain the same code for the Vita version. While new NPC types are added to the game, the core systems powering your opponents are identical and, as Porter puts it, "all previous behaviours are supported out of the box". The PS3's SPU-driven AI code again found a home distributed across the three available Vita ARM cores, though a special emphasis was placed on optimisation.
"Raycasts are the major contributor to the CPU load, and even across cores we can't compete at a processing level with the SPUs," says Porter.
"To mitigate, performance has been found in rationalising both the number of casts made and their lengths, in places. Atop, we've also found optimisation at an algorithmic level within the higher-level behaviour: to avoid the need for some of those expensive calls to be made in the first place."
Other optimisations were made, carefully nipping and tucking at the algorithms to increase performance with only minimal impact to the effect on-screen, as Porter explains:
"We set a budget on the number of NPCs that should be active at any one time. This has steered the design team's level choreography from almost day one and to support that effort we've extended KZ3 scripting systems to allow tight management of the active/inactive AI levels. Through the level designers' hard work on the scripting, we hope that any delta between the platforms is invisible."
In theory, it's also possible for many of the PS3 version's graphical assets to be ported over with little or no compromise to the Vita game. Part of the Killzone 3 engine's multi-platform support allows for the export of assets into different platform targets.
"If an asset converts - and the majority do - then we should be able to run them on Vita," explains Porter. "That's the idealistic picture, and whilst we exploited it considerably at the early stages to get prototypes up and running (with models, environment building blocks and characters), the reality is that we've tuned most of that content against the memory and performance constraints of the platform."
"Virtually every post-process effect from KZ3 was ported over. We have a full depth-based colour correction system, depth-of-field, film grain, bloom, a new volumetric fog system and more."
A recurring theme in Vita games development has been a compromise on resolution, with almost every technologically demanding game seeing a reduction in detail from the Vita's native 960x544. For example, Criterion's Need for Speed: Most Wanted renders natively at 640x368, using multi-sampling anti-aliasing (MSAA) and the hardware's beautiful screen to help mitigate matters. Guerrilla Cambridge's approach was uncompromising - the developer wanted to hit native resolution, retain anti-aliasing and still maintain the 30fps frame-rate of the PS3 games.
"Graphics is all about trade-offs; in Mercenary, we put a very high emphasis on image quality. IQ is degraded in many ways (banding, temporal aliasing, texture filtering, etc) but by far the biggest impact is the resolution and AA method," says software engineer Graham Aldridge.
"From day one we were running at native resolution. This had a huge impact on the development of the engine and the trade-offs made - it's very easy to want an engine to do everything. For example resolution, anti-aliasing and image quality were major factors in the decision to use a forward renderer over a deferred renderer." Memory and bandwidth control was a major challenge for the Guerrilla team, and dynamic resolution scaling was introduced in order to maintain frame-rate in action-intensive scenes.
Killzone: Mercenary's take on dynamic resolution scaling
"The engine supports native resolution and lower (quarter) resolution rendering for fill-rate heavy items such as particles. Dynamic resolution scaling is supported for the main scene, reducing the horizontal resolution incrementally. However, it has no effect on the lower-resolution rendering or post-processing systems," clarifies Aldridge.
The engine also engages the dynamic resolution modes in specific conditions where the player is less likely to notice the impact to image quality. If the camera is still, the engine maintains native resolution and frame-rate is allowed to drop - the player is not engaged with the game and visually the lower performance is not so noticeable. It's a really impressive technique, allocating system resources to what matters most at any given moment.
"It is our belief that players are less likely to notice the temporal effects of the resolution reduction when they are in motion (indeed, many of those effects are likely to be shorter-lived if one is moving through the scene). When static, compromises to image quality are likely to be much more noticeable than frame-rate drops - particularly in screenshots," Matt Porter says with a smile.
"Dynamic resolution scaling is supported for the main scene, reducing the horizontal resolution incrementally. However, it has no effect on the lower-resolution rendering or post-processing systems."
But the key to sustained performance is all about optimising for the platform, as Graham Aldridge explains.
"For scene geometry and effects, we did a lot of low-level shader optimisation," he reveals. "On mobile hardware, you can measure the difference one cycle makes. This meant our common material models had to be very efficient. For example, dynamic point lights costs us eight cycles on the fragment program for two lights (with normal mapping). We go as far as using cubemap normalisation when we aren't texture limited!"
It was in the interests of performance that Guerrilla Cambridge chose to drop the deferred shading lighting algorithm that was so successful on the PS3 Killzone outings, as Guerrilla senior artist Matthew Birkett-Smith reveals:
"A deferred solution was investigated initially but in the end we opted for a forward renderer," he says. "This was a trade-off that allowed us greater shader variety and a more sophisticated light-mapping system but did slightly restrict us on the number of dynamic lights we could support. It was also one of the factors that allows us to hit full native resolution while retaining MSAA support."
Deferred shading allows the PS3 Killzone games to operate with hundreds of light sources in any given scene. Guerrilla Cambridge chose to build upon the pre-baked lightmapping elements of the Killzone engine, and actually added other features.
Advanced lighting: HDR, dynamic exposure - and more
"One huge change for us was the addition of full HDR lighting support in our pipeline. Another large change was to store both the dominant direction of light on any given surface and also how strongly directional or ambient this lighting is," says Birkett-Smith. "This made it possible to calculate a very efficient normal-mapped diffuse and specular lighting model for our materials. Virtually all materials in the game contain diffuse, normal maps and specular, and having an efficient solution to show off our materials was a very big win for us.
"As well the colour, HDR brightness, dominant lighting direction and directional strength of lighting we also pre-calculate our sunlight coverage. This is used in-game as a signed distance field to give us sharper edges on our sunlight. For actually lighting levels offline, we use an internal Sony solution called NOVA - you will also have seen this used in many other internally developed games like Uncharted 3, The Last of Us and of course Killzone 3."
The revised tech, working in combination with the small screen, gives the appearance of a full-blooded Killzone game, even though one of the tentpole systems that defined the look of the PS3 game has been completely removed. The horsepower freed up by the new lighting system allows for some exceptional effects work.
"At a game level... many of the core components of a Killzone game remain untouched and worked pretty much out-the-box on PlayStation Vita."
"One of the largest post-related effects for us was the use of automatic exposure adaptation and how this works together with our HDR lighting system," Matthew Birkett-Smith reveals. "In-game we are constantly, subtly changing the exposure levels of your view to keep the screen luminance balanced. With this we can transition smoothly between very bright exteriors and dark interiors while keeping strong contrast and detail at all times. The end effect should feel fairly natural to the player but from a lighting point of view it lets us really push lighting levels between different areas and make the whole world feel more grounded."
Other advanced post-process effects work demonstrates that being able to directly address the Vita's graphics hardware can result in some impressive, albeit computationally cheap, wins.
"Another large addition was the introduction of screen-space reflections into our material pipeline," continues Birkett-Smith. "This lets materials reflect in real time other visible areas of the game world and really adds to our material definition. This effect was actually relatively cheap to implement and we use it to varying degrees in our materials, it probably being most notable on flooring and our first-person gun models."
But many of the technological underpinnings of the Killzone 3 engine are still present and correct, ported in their entirety across from PS3.
"Aside from this virtually every post-process effect from KZ3 was ported over," Birkett-Smith concludes. "We have a full depth-based colour correction system, depth-of-field, film grain, bloom, a new volumetric fog system and more. On top of this we also implemented tone-mapping and local contrast adaptation, both of which works really well with our HDR lighting model."
But at its heart, despite the technological cutbacks and the introduction of additional rendering features, Mercenary is clearly a Killzone game - one that complements the existing games in the series in much the same way that Golden Abyss retains the essence of the classic Uncharted titles. Before we got hands-on with the final game, the preview code we checked out felt very much as though it was tonally in tune with Killzone 2, rather than its successor, though as you can see from the captures on this page, the scope of the game is clearly significantly beyond that single-level teaser we previously revealed.
"I like to think that what makes any Killzone game a Killzone game has always been there at the core of any of the titles in the series, but that each one is driven with slightly different things in mind," says senior producer Mark Green.
"Mercenary is no different, and with much of the story spanning the action in Killzone 2 I suppose it was always going to be a closer match than perhaps any of the other games. This is true visually, but also I suspect that there was a subconscious influence involved: Killzone 2 was such a cornerstone in development on the PS3 that it has influenced us, not just in terms of visuals, but also in terms of handling and feel. I still believe that there are enough new tweaks in Mercenary to make it feel entirely fresh and new, but with a homely familiarity that is comforting and immediately accessible."