PlayStation 5: what to expect from next-gen console ray tracing
Gran Turismo 7, Ratchet and Clank and Pragmata lead the charge.
With nigh-on 30 titles revealed at last week's PlayStation 5 software showcase, there's certainly plenty of material for Digital Foundry to mull over in its wake - but one of the biggest and most pleasant surprises was the range of games from both first and third party developers using the machine's hardware accelerated ray tracing technology. But what was actually shown that we can say for sure was using hardware RT? What kind of effects were featured and how do they function? And perhaps most crucially, going forward, what does the software showcase tell us more generally about how ray tracing will be deployed in the next-gen console era?
To begin with, I think it's important to give credit where it's due. Hardware accelerated ray tracing only made its debut in a shipping video game in November 2020, when DICE updated Battlefield 5 with its sub-optimal but still impressive RT implementation. It was updated months later with radical performance upgrades and was just the first of many RT-supported titles. However, the point is that for consoles to embrace this emerging technology so quickly and to be deliver so many examples of hardware RT in an initial software reveal borders on the miraculous, especially given how computationally expensive and complex ray tracing is.
How expensive? We learned that quickly enough on PC two years ago with RTX-supported titles: a single full-resolution ray tracing effect like reflections at 4K on the most powerful GPU available can drop frame-rates by significant double digit percentages. Other effects like ray traced shadows can be less intensive, but still significant. The point is that console hardware design is based around bang for the buck, meaning that developers will need to be careful in how RT is used to keep resolutions and frame-rates reasonable. In the first wave of PlayStation 5 titles we saw, we got a few indications of how the developers are adapting to this new rendering paradigm.
Gran Turismo 7 is as good a place as any to begin. The use of hardware RT here is undeniable - with reflections deployed on a number of the game's surfaces and on the car bodies themselves. Today's reflection tech of choice - cube-mapped reflections, as seen on Forza Horizon 4, for example - capture facsimiles of the environment then map them onto the bodywork, missing self-reflections and presenting in a perspective-incorrect manner. GT7 is on another level: environment reflections are properly mapped onto bodywork, all cars and indeed the player's car are also reflected, as are elements that are off-screen. It's a significant upgrade to the materials work on the player vehicle, while materials like glass seem to receive ray traced reflections too. It's also the first Gran Turismo game we've seen where the rear view mirror includes the car interior and indeed the driver - by comparison, GT Sport's equivalent presents like a camera stuck to the back of the car.
The game renders at what looks like native 4K resolution, so achieving these results at 60fps requires some innovative approaches to ray tracing. First of all, reflection resolution looks to be quarter resolution, cutting down the RT workload significantly. On top of that, a curious element is that moving edges in reflections exhibit sawtooth lines, much like checkerboarding. This makes me wonder whether interlacing or checkboarding the reflections is another push by Polyphony for higher performance - we saw a similar technique in Metro Exodus's real-time ray traced global illumination on PC.
The strategy does seem to be about getting the biggest visual return possible from limited hardware RT resources. In real life, unless an object is a perfect mirror, the reflection on a slightly rough surface becomes more diffuse and possesses some darkening based upon the camera angle in relation to the reflection. This glossy reflective look requires more rays to be traced, as the ray hits making up the softer part of the reflection have less coherency or more random directions, making the effect more expensive to process. One way to optimise around this is simply to ignore the effect and deliver perfect mirror-like reflections, then using cheaper post-processing to simulate the more expensive effects afterwards. Sharpness in reflections is consistent in GT7 on PS5, suggesting a similar optimisation: reflections do not change their level of sharpness within a single material type, only across different materials.
It was also heartening to see ray tracing showcased and name-checked by Insomniac in Ratchet and Clank: Rift Apart, where the developer told us that ray traced reflections are used on Clank himself (witness the subtle reflections of his body captured on his chin). However, watching the footage, it really looks like ray traced reflections are used on a number of the most shiny surfaces in the game, such as the metallic floors seen in the gameplay segment. Unfortunately there is no high quality video for this part of the presentation to take a closer look at, but even at this low quality, there are some interesting optimisations in play.
Much like GT7, it appears that there is a uniform level of reflectivity across the entire surface, with no contact hardening. Another interesting aspect is that a number of objects look to be excluded from the reflections. I am not sure if these are bugs from early footage, or a technique for saving on performance by having fewer objects in the BVH acceleration structure. Certain dynamic elements also appear to be removed from the reflections, such as debris from destroyed objects. The reason why this optimisation might be present in this early footage is because moving objects or rapidly changing objects in ray tracing need to rebuild the accelerated ray tracing structure in real time on the GPU - and this is a separate cost on top of the computational effort in tracing the rays or shading them. As a rule of thumb, dynamic objects are far more expensive to calculate in ray traced reflections compared to static ones and what we're seeing in Ratchet and Clank bears this out.
In addition to first-party developers, independents also look set to use hardware accelerated ray tracing as seen in games like Stray by Blue Twelve. This game is based on Unreal Engine 4, which has its own baked-in RT feature set (as discussed recently in our Ghost Runner coverage). Based on what we've seen of UE4 so far, its ray tracing features are of a very high quality but they are computationally expensive. For example, UE4's reflections have darkening, stretching and associated physically accurate aspects. The Stray trailer follow suit - the scene with the barber robot and his customer shows highly precise reflections in the mirror in front of the characters, including off-screen elements (so it's not using standard screen-space reflections). And since the mirror is not perfectly clean, the clarity of the reflection changes with the material surface, which looks excellent. As we go outdoors, this attention to detail plays out in a much wider, grander scale and really looks fantastic. This game may also have other RT effects like ray traced shadows, but equally, they may well be precalculated or 'baked' - an area where UE4 has some excellent tools.
That uncertainty is something that applies to other games we saw, like the Bluepoint remake of Demon's Souls, where the PlayStation blog reveals that the title has ray traced shadows. However, it's unclear if this applies to indirect or direct shadows and nothing in the trailer or the accompanying press shots gives anything away. There's much to discuss with Demon's Souls in other areas - and we'll be looking at that in more depth soon. It's also worth noting that Insomniac has discussed ray traced reflections in Marvel's Spider-Man: Miles Morales, but again, there's nothing concrete in the tiny teaser Sony chose to reveal. In both cases, we can't wait to get hands-on.
The final example of hardware accelerated ray tracing we picked up on concerns Capcom's Pragmata, based on a presumably beefed up version of the excellent RE Engine, which has delivered some stunning games in this console generation. In this title, there are two visible types of RT in play. Inevitably, it begins with ray traced reflections: they're obvious in the shot of the child's eyes, to the point where electronic billboards reflected in her eyes change in real-time within the reflection itself. And if you look close enough, you can even see how her eyelashes and elements of her own face are reflected in a distorted manner. Ray traced reflections are amazing for this reason as they offer infinitely accurate detail no matter how close the camera gets.
An interesting aspect of these reflections is that they function a lot like those seen in Battlefield 5, where screen-space reflections (mirroring detail rendered on-screen, essentially) are merged with RT reflections in the same scene. If you use low or medium quality RT in BF5, the game increases the roughness cut-off for reflections to save performance, but SSR is still applied so rougher surfaces still get some level of real time reflections - even if they are just in screen-space. This is exactly what Pragmata does in this trailer, where the ultra sharp and cheaper reflections in the character's eyes, or the puddles on the ground are achieved using ray tracing. However, the rougher reflections on the street are delivered mostly through SSR, so you can see such reflections moving in and out of existence when the source detail is obscured. It's a very smart combination and again delivers a lot of bang for the buck from limited resources.
The second ray tracing effect in this trailer is even more interesting and takes the form of some form of ray traced global illumination, with some subtle but remarkable effects as light bounces around the scene. As the astronaut character walks in front of a yellow taxi, bounced light from the vehicle is seen on the astronaut's suit. As it lifts its hand, you can even see an indirect shadow in the bounced light from their arm across the chest - it's a remarkable level of fidelity. There are some artefacts though: if you watch the footage, you will notice how this indirect lighting on characters is especially grainy, as if the light is made up of tiny stippled points. This is generally how real-time ray tracing looks when it is not aggressively denoised. Certainly on early footage, I don't mind seeing artefacts and lower levels of precision - it just adds to the authenticity.
Pragmata is scheduled for 2022, so of all the games featured in the showcase, it's likely the one earliest in development, so much may change - but in my view, this combination of RT effects delivers the most impressive utilisation of ray tracing seen at the PS5 event. Perhaps it points to the level of RT utilisation we will see a few years down the line - using smart optimisation for keeping ray traced reflections more reasonable in performance terms, while having enough rendering juice left over for game-changing real time global illumination.
Ultimately, as a ray tracing evangelist, I came away more than satisfied from the PlayStation 5 reveal - mightily surprised, even. This console generation has used the same kind of faked effects for so long that we've got used to them, and we don't even notice the errors and fakery in many scenarios. For example, nobody seems to question why they don't see the player character reflected in mirrors any more (even the original Duke Nukem 3D managed this!). Ambient occlusion and global illumination has evolved and looks good, but the overall effect falls short compared to what we've seen from ray traced implementations - the comparison between approximated and RT-simulated is stark.
From my perspective, the good news is that the PS5 software showcase demonstrated that game-makers aren't ignoring the new hardware and banking the more obvious performance wins from a more powerful GPU - they're experimenting with RT features in the here and now, and it looks like we may see some of the fruits of their labour at launch. If you'd told me that this would be the case at the beginning of 2020, I'd have struggled to believe it. Combined with the fully path-traced version of Minecraft we've already seen running on Xbox Series X, it's great to see console RT hardware backed by immediately visible results - and as a brand new tool in the developer's toolbox, I can't wait to see what happens next.