How today's games could look and run better on future consoles
Xbox One X provides a compelling proof of concept. Digital Foundry talks to Microsoft's ATG team.
Since the arrival of the Xbox 360 and PlayStation 3, games have skewed towards multi-platform projects, but could it be that we are now moving into a period of multi-generational development? After all, we now have developers targeting console hardware with graphics power at the base Xbox One's 1.3 teraflops, right the way up to Xbox One X's 6TF, with a brace of PlayStations sandwiched in between. Techniques and technologies have emerged to help bridge the gap - and a knock-on effect is potentially improved performance and image quality on the games consoles of tomorrow.
Yesterday, we published our discussion with Bill Stillwell of Microsoft's backwards compatibility team, but also present was Martin Fuller of the Advanced Technology Group: "My team works a lot with developers when they're making the games, whereas Bill's working with the games after they're made, so we are always kind of looking towards the future and helping developers adopt new technologies," he explains.
ATG provided the reverse tone-mapping technology that allows Xbox 360 games to run in HDR on Xbox One X, but there is more crossover between the two teams than you might imagine. A whole new back-compat opportunity arose with the arrival of the X, a chance to take the existing catalogue of Xbox One titles and run them better on the new machine. Smoother performance, improved texture filtering and faster loading are the main upgrades offered at the system level, but a small range of titles deliver all of that and more thanks to dynamic resolution scaling, a technique ATG has championed to game-makers.
"We obviously knew we were doing Scorpio a long time before we told developers who were doing Scorpio, so we knew that with dynamic resolution we would be able to scale up and run at a maximum resolution and that would be a huge benefit to people playing unpatched games," Fuller says, and while the plus points are clear for Xbox One X and whatever hardware may follow it in the future, the principal benefits of dynamic resolution are geared more towards getting the best out of today's hardware.
"There are a lot of advantages to dynamic scaling - so obviously one of the chief ones is that it's better to drop resolution than it is to drop frames, so we're very passionate about trying to keep the frame-rate up, but also trying to use a hundred percent of the GPU all the time," he continues.
"So, if you have a game that's just running at a fixed resolution all of the time, it's got to maintain an overhead for its kind of worst case scene, so if you're not in the worst case, you're wasting GPU power, it's idling. With dynamic resolution, you bump up to max res and you get a great image, but maybe when a firefight kicks in, you're running a little lower resolution - and nine times out of ten, when you're playing a game you don't even notice. So, we've helped a lot of titles with kind of best practices to get going with dynamic scaling."
Whether it's down to Microsoft's evangelism, or simply momentum in developers adopting the technique after seeing great results elsewhere, the bottom line is that dynamic resolution is prevalent across the majority of the most recent triple-A titles. This quarter we've seen different flavours of DRS rolling out on games including - but certainly not limited to - Assassin's Creed Origins, Destiny 2, Star Wars Battlefront 2, Wolfenstein 2 and Call of Duty WW2. We put it to Fuller that based on what we've seen with Xbox One X running older DRS games, these titles exhibit some level of future-proofing for potential hardware to come.
"Absolutely! There's another significant advantage to DRS, which is that we're now in the world with consoles that we've never been in before. There are many different console SKUs out there and one of the things that DRS helps developers do is balance their game for all these different machines," he says. "So, on PC, you have a ton of options and it's up to the gamer to choose their options and kind of choose their frame-rate. With DRS, they can balance for the S and they can balance for the X, and they're not having to quite so finely tune each one of them to make sure they utilise the GPU 100 per cent of the time. Before, they only had one Xbox platform to do that on. So that's another significant benefit for developers."
From our perspective, Titanfall 2 is probably the best example of dynamic resolution scaling in terms of achieving solid 60 frames per second gameplay by effectively trading pixels for frames. However, what we've found is that most games operate with upper and lower bounds - and when you're at the lower bounds, frames can still be dropped. When running on Xbox One X, we've noted that games like The Witcher 3, Battlefield 1 and the Doom 2016 reboot all benefit from much smoother performance as well as maxed resolution. DRS combined with much more powerful hardware can provide a transformative impact on the quality of the experience.
Hand-in-hand with DRS has been the emergence of temporal anti-aliasing, or temporal super-sampling - using previously generated frames to improve the quality of the current one. "Temporal anti-aliasing, well, you actually use the right word there, temporal super-sampling. That's a really great technology," enthuses Fuller. "So, for a long time, we've been kind of fighting the jaggies and temporal anti-aliasing was a cheap way of injecting additional information, compared to FXAA which doesn't have any additional information - it does a great job but it doesn't have any more information to increase the quality of the image."
TAA is a complementary technology to DRS, often used in tandem, and it's difficult to argue with the results. On the base consoles, image quality can look a touch blurry when the dynamic scaler hits its lower bounds, but it's an easy trade to make compared to the alternative of shimmering jaggies, lower performance or a combination of them both. Although marketed primarily for its 4K credentials, Fuller is excited about the combination of Xbox One X with a full HD display.
"With temporal super-sampling, you're bringing in information from the last frame, and one of the really exciting things we do with an X when we're connected to a 1080p television is super-sampling," he says. "I've been doing this a long time and you know, I never thought I'd see a console powerful enough, but super-sampling is the gold standard of anti-aliasing and it's exactly what the film industry use, it's the reason you don't see jaggies in films. So, anyone that's connected their X to a 1080p television is going to get much higher image quality. They're really going to see a step up with the X and then if they upgrade again from 1080p to 4K, they're going to see another step up. They're going to get the step up in clarity."
At Digital Foundry, we've been firm advocates of SSAA for 1080p display users, which made the inconsistency of support in PlayStation 4 Pro titles puzzling and often frustrating. If a developer creates specific modes with different performance characteristics, all modes should be available to all users regardless of the display they use. This is what Microsoft promised to us when we visited Redmond in March and the firm's method of delivering it is pretty straightforward.
"The way we do that is incredibly simple," says Fuller. "We just don't tell developers what television they have connected."
From our perspective, it's been fun to review the back catalogue of Xbox One titles, identifying the games that would stand to benefit the most from the out-of-the-box improvements Xbox One X supplies as standard. Doom is our best example of a game where temporal super-sampling and dynamic resolution scaling produce a significantly improved experience. However, Martin Fuller points to Mirror's Edge Catalyst as an example we missed in our initial testing. We had DICE's game pegged as a native 720p title when it launched, but it turns out that DRS is in place with Xbox One X effectively delivering 900p now at a significantly higher frame-rate.
And with so many of the latest games now utilising DRS, TAA - or both - and with Microsoft's commitment to older games running better on future hardware, we can't help but wonder what the true next-gen Xbox will bring to the party. It's not unreasonable to expect see system-level improvements similar to those seen in Xbox One X, but equally, with dynamic resolution now commonplace, there's built-in scalability in a range of games now, and their full potential could be unlocked in the years to come. Maybe Assassin's Creed Origins' dynamic 1700p-1800p general range will lock at full 4K. Maybe the move to Ryzen CPU cores paired with a big GPU upgrade will see Wolfenstein 2 hit a fully uncompromised 2160p60. While the next generation is still some years away, some of the impressive back-compat results seen on Xbox One X act as a pretty compelling proof of concept for what could be.
Microsoft's work in taking forward the existing Xbox library on more powerful hardware has surely set a level of expectation for core gamers, and it poses an interesting question: to what extent will Sony respond with its own next-gen hardware? This week, SIE announced an installed base of over 70m PlayStation 4 users. On the one hand, this impressive figure suggests that there's still a few years left for the current console generation - but on the other, surely Sony won't want to leave those users behind when its new console eventually drops?
Make no mistake: the success of the true next-gen consoles will be built on games that set new standards and deliver new experiences, but at the same time, Xbox One X - not to mention the popularity of Sony's own Boost Mode for PS4 Pro - has demonstrated that the ability to play improved versions of our existing library of games is valuable and compelling. But our conversations with Microsoft have demonstrated that getting to this point hasn't been easy, requiring a highly skilled engineering team and over 100 testers ensuring that everything just works. But what's fascinating here isn't just the focus on backwards compatibility - but also the notion of building future scalability into the console games of today.