Digital Foundry vs. 3D Gaming

Sony is planning to upgrade the firmware of the PS3 to officially support the new range of 3DTVs, and every PS3 model released to date will be made compatible. The new firmware upgrade itself is only one half of the equation and in itself is not a magic bullet that solves all the rendering problems facing developers. Behind the scenes the system update will be mirrored by a new SDK revision from Sony that should lay the groundwork for developers to use 3D the way the platform holder has in its impressive WipEout HD and GT5 demos. The chances are that these tools have already had a limited release to selected developers: how else could Super Stardust HD from Housemarque already be running in full stereo 3D?

More than that, a firmware update suggests a retooling of the PS3's system software at the very lowest level: the inference being that PS3 as a whole is being upgraded for the migration towards 3DTV. It has already been announced that the PS3 will become compatible with the new range of 3D Blu-ray movies, which means that the console is going to require updating to match the HDMI protocols used by the requisite displays.

Here's where things get interesting. The HDMI 1.4 spec features support for a wide-range of 3D techniques, including some of those supported in Avatar. However, it is not just limited to the choices Ubisoft made; there are two other techniques of real interest here that represent a leap in quality, and that would only be possible with a firmware upgrade.

First up is the notion of 2D plus depth. This is something we've covered before in a previous In Theory piece: by beaming over the standard 2D image in combination with data derived from the console's Z-buffer, it is conceivable that some kind of 3D image could be rendered.

However, the use of the Z-buffer on its own presents several difficulties as you are almost certain to be missing crucial visual information. Let's say you're looking at the edge of a wall. One eye will see only the wall, while the other should see other detail from behind it. Using the Z-buffer alone, you'll never see beyond the edge. Transparencies, or other effects that don't use the Z-buffer won't be rendered in 3D at all. Developers also need to double (or triple) buffer their output, and the Z-buffer doesn't tend to work like that.

So while we expect to have something special cooking from within Sony R&D utilising the new HDMI protocols, 2D plus depth information probably wouldn't cut it, and crucially, the developers would have little control over how the final image would look. Looking at the HDMI spec, it doesn't seem as though many other options are available, but right there is the use of side-by-side rendering in both full and half modes. We've seen the half-mode in both Invincible Tiger and Avatar, but full side-by-side mode is something completely different, and much more exciting. This would have the PS3 working on the principles of a 2160x720 or 1280x1440 resolution - essentially two complete 720p frames working side-by-side, beamed over to the 3DTV at 60Hz, with the frame then split into two and sent to each eye.

This seems like a likely method for adoption within PS3 and it is borne out by the choice of games that Sony has demoed: WipEout HD runs at a minimum of 1280x1080 with its dynamic resolution in 1080p mode. Similarly, both Super Stardust HD and GT5 run natively at 1280x1080. The showcasing of these particular games suggests that there's horsepower left under the bonnet for a larger-than-720p framebuffer. The nature of stereoscopy itself means that the process of creating the discrete views for each eye needn't be too onerous: while two views need to be rendered, the actual differences between the frames are fairly slight thanks to our eyes being so close together: the frame is prepared once, with one command buffer, and then after it's used to generate the first image the buffer is patched with the other eye's projection matrix and it is rendered again. Compute-heavy stuff, such as shadowing, only needs to be generated one time.

In short, you can't expect the new PS3 firmware and SDK to make the challenges of rendering discrete images for each eye completely go away - all the really clever stuff still needs to be done by the developer. However, certain games will be more disposed to being patched to support it and future games can be designed with the capability in mind. Should 3D take off in a meaningful way, the art of its implementation will pose interesting challenges to developers.

"The reverse re-projection surface pixel cache system I used in early Trials HD development builds could really easily support stereoscopic 3D rendering," RedLynx's Sebastian Aaltonen posted on the Beyond 3D Forum. "Stereoscopic views would need only very little extra processing power as the geometry passes are really cheap (they are basically just simple dual projected texture geometry with no light calculation, no shadow calculation etc). Stereoscopic rendering is really a poster child for all pixel reusing techniques."

Even if the full side-by-side mode isn't viable, Avatar's various implementations, based on the use of two lower-resolution images images combined into one conventional framebuffer, will still work too; they just won't have quite the same pristine edge as the full resolution approach.

So, where does this leave the Xbox 360? It's fair to say that Microsoft has shown no sign of supporting the format whatsoever, while Sony obviously has a huge vested interest in the format being a success, factoring in its investment in 3D displays and movies. Whether the HDMI 1.2 output of the Xbox 360 could be patched with any of the full-resolution stereo 3D modes remains to be seen, but it's fair to say you shouldn't be holding your breath. However, certainly in theory, a side-by-side 720p buffer is conceivable, and Avatar 360's advantage in 3D performance over its PS3 counterpart is obvious even without recourse to FPS analysis, suggesting that the horsepower is there too: at the end of the day, there are few unique effects that one console can achieve that the other can't match. The only question really is whether the Xbox 360's HANA chip could be firmware-upgraded to interface with the new displays.

But in the here and now, the ball is firmly in Sony's court and while few people may actually end up investing in the new screens before the advent of the next-generation consoles, the fact that the support is there within the PS3 adds to the kudos of the platform, and gives the sense that the console is evolving in line with trends in cutting-edge consumer electronics. There's a definite momentum towards 3D technology going into 2010, with the success of James Cameron's Avatar propelling the profile of proper, stereoscopic 3D very much into the mainstream. Needless to say, PS3 could play a very important part in that.

The various stigmas associated with 3D are beginning to fall by the wayside. The headache-inducing red/blue anaglyph 3D is already a thing of the past and we can expect the active shutter glasses of the first generation of 3D screens to be replaced in time with a superior polarised experience (similar to the effect seen in the cinema), and this in turn will give way to self-contained 3D displays where no glasses will be required at all. It's going to take a look more work and available media to convince people to upgrade - particularly when the chances are you've only recently bought an HDTV - but the fact is that 3D works, and the results can be extraordinary.

In terms of gameplay, expect to hear - and see - much more at March's GDC...