The Anti-Aliasing Effect

One of the most important elements in overall image consistency within console games has to the implementation of anti-aliasing - the process of smoothing edges and mitigating or eliminating the dreaded "jaggies". Both PS3 and Xbox 360 have AA technologies built into their respective GPUs, but as the console generation progresses it's becoming clear that for many games there are simply not enough system resources available to anti-alias images using traditional techniques. Developers are beginning to look beyond the hardware solutions and come up with their own methods. Some work well, some look awful, but it's all evidence of game-makers looking to eke out as much performance as they can from the current fixed architectures.

Before we continue, it's worth pointing out that aliasing doesn't just apply to dodgy-looking edges; it can also extend to other artifacts when rendering with texture and shaders, including moiré patterns and specular aliasing. However, the process of edge-detection and blending is the basis for this piece.

From a hardware perspective, edge-smoothing on Xbox 360 should theoretically be entirely "free", with the GPU doing the work for the developer. However, limitations in the on-die eDRAM memory stop that from being the case unless you opt for a sub-HD resolution (cases in point: Ninja Gaiden 2 and Modern Warfare 2). The memory requirement means that additional processing is required to implement anti-aliasing on 360 at 720p. For the PlayStation 3, running at a disadvantage in terms of bandwidth and memory, handling anti-aliasing is more difficult - hence this element being cut down in so many of the cross-platform games Digital Foundry looks at for the Face-Off comparison pieces.

PS3 is curious, however, in that it has hardware support for two widely used AA techniques. We've discussed MSAA already, but quincunx AA is the other most frequently implemented technique. Unique to the NVIDIA hardware, it uses approximately the same amount of resources as MSAA but produces superior edge-smoothing at the expense of adding a blur to the entire screen. The use of quincunx and the impact on overall image quality varies game by game - intricately detailed textures will suffer much more than a more flat, anime style of art. However, this Assassin's Creed comparison of 2x MSAA up against quincunx demonstrates both the edge-smoothing advantages and the detail blur.

Colin McRae: DiRT 2 is an interesting example of how quincunx can be used to produce a look very close to the full-on 4x MSAA that can be extracted from the Xbox 360's Xenos GPU. The range of post-processing effects used in the EGO engine seem to suit quincunx as a "lighter" alternative to the 360's more precise but more memory-intensive solution. In short the game achieves some level of its realism from the motion blur employed, and in this case quincunx's less impressive side effects are integrated into the overall art scheme. Killzone 2 does the same thing.

The memory and bandwidth implications of the established hardware techniques can be seen as rather heavy, with game-makers looking to redirect that GPU effort elsewhere. That being the case, we have seen a number of customised anti-aliasing solutions in games such as Overlord II and Call of Juarez: Bound in Blood. We've also seen some fairly primitive edge-smoothing solutions that simply seek out the polygon edges and then blur them. Brutal Legend is a case in point.

"We have elected to use our own solution because hardware MSAA cannot be used while rendering using multiple render targets," says Call of Juarez lead shader programmer Maciej Jamrozik. "Another reason against going for hardware MSAA was the fact that it only smooths edges of geometry but not edges generated with alpha-test. The latter are prevalent in our titles, especially through the vegetation system. In addition to that, hardware MSAA requires extra memory which is a priceless resource especially on consoles. Finally, on Xbox 360 there’s the issue of buffer resolving which also reflects upon final performance."

Developer Techland went for its own custom anti-aliasing solution that copes with transparent objects, and has a lower impact on system resources while at the same time producing an effect fairly close to hardware MSAA. Not only that, but the support for multiple render targets means that their system would work in a deferred rendering engine, as seen in Killzone 2. Guerrilla Games actually added a more traditional forward renderer on top of its deferred tech partly to accommodate effects like anti-aliasing. The Call of Jaurez makers are in effect cutting out the middle man.

The recent release of The Saboteur by Pandemic revealed one of the most interesting uses of custom anti-aliasing seen to date. While it's said that the technique used was a simple case of finding edges and blurring them, the overall result is a huge increase in image quality over other examples. The algorithm runs on the PS3's SPUs, locating edges through luminance and then applying multiple passes of blending to give results that look absolutely phenomenal.