Forza Motorsport on Scorpio: the full story
How Turn 10's software helped to shape the next Xbox.
Just prior to Digital Foundry's visit to the Microsoft campus, Xbox's Senior Director of Console Marketing, Albert Penello, asked me what I needed to see on the day, what I thought was critical to the story. The answer was obvious and immediate: beyond the specs and the tech deep dives, I needed to see software running on actual hardware. Promises had been made concerning Xbox One engines scaling up to 4K and as compelling as the tech talks would be, only a real-time demo could really validate the claims made for the machine. Penello sat back confidently, nodded, smiled and the conversation moved on.
Less than 24 hours later, I'm talking with Chris Tector, Studio Software Architect at Turn 10 Studios. In common with every Scorpio presentation I've had that day, there's a sense of excitement in the atmosphere. He's genuinely thrilled to share the studio's role in Project Scorpio's development and yes, I will be seeing Forza running in real-time on the new console.
However, the 'ForzaTech' demo I'll be seeing won't be Forza Motorsport 7 - software announcements aren't typically tied into hardware deep dives - but clearly, Turn 10 and the Xbox team are confident that final software will match or indeed exceed the quality of what I'll be reporting on.
"ForzaTech is really the place to let us prove out platform features like Direct3D 12 [on PC] and also lets us help with internal things that nobody else gets to see - in this case, the modelling that happened with Scorpio," shares Tector. "We were really able to help prove that out... Our part in it - and an important part for us, because it's a two-way thing - we wanted to make sure that the hardware was going be able to do what we wanted, what we had as a vision, what we wanted in Forza next, but we also wanted to make sure that however it's being proved out, that the promise was solid, that the model made sense to us."
As we've already revealed, Microsoft's innovative technique in defining, refining and customising Scorpio's silicon design came from taking existing Xbox One game engines into the optimisation tool PIX (Performance Investigator for Xbox) and using that data to shape the design of new processor. Turn 10 were able to do more: to tweak the game engine as they saw fit, handing over multiple sets of profiling data for the hardware team to run on their emulators, way before any actual hardware was spun.
"We provided a ton of data with ForzaTech, where we actually rendered different stress scenes at different resolutions - 720p, 1080p, 4K - and then stressed different points in the engine: anisotropic filtering, multi-sampling, pushing heavy LODs through, just to try to get a feel for where the different bottlenecks where," says Tector.
"And by knowing that we had created those scenes and rendered them at each of those cuts, we were able to prove that the model actually scaled, because we could say that the model predicts that if you took this 720p scene and went up to 1080p with these options, you should get X performance. And we could actually check that against what we'd actually done on an Xbox One. That really gave us the confidence at the time of the model that we could actually scale up, and importantly it proved out the whole model."
This profiling was just one set of data that the Xbox hardware team had to work with. More data was coming in from other titles, and the scaling results in the move to 4K were looking consistent. "All the PIX captures and analysis and simulation they did proved it out for everyone, not just the people who were going to target 4K60, starting from a point at 1080p60 [like Turn 10] but even the people who haven't gotten to that point yet," continues Tector. "They have other reasons that they aren't going for native resolution maybe and so they've made other trade-offs in their engine and they have other bottlenecks than we would. I think it was great that the model hit such a broad set of different rendering types, it really helped prove it out."
Additionally, by working directly with the Xbox hardware team, Turn 10 is able to provide invaluable developer input, way ahead of time. "Importantly it feels like the right stage to be at, especially this early. It's like, we can evaluate it early before other studios get it and before they all have to go through that process as a group," enthuses Tector. "We can do it as a first party and help prove out the platform because we're not only trying to make the platform the best one for us, we're trying to make it the best one for all game developers."
"I can't stress enough how important it is for our platform team to have an asset like Turn 10 and Chris and his team," chimes in Kevin Gammill, Group Program Manager of the Xbox Core Platform. "Not only do they help provide the modelling, but they also keep us honest with the developer perspective as well as the consumer perspective in doing the right thing for our customers, so it's fantastic partnership."
"It's been so powerful to have a developer voice in the room. I'm sure there has been throughout history, but it was just a relief," adds Tector. "It said a lot about the Xbox organisation to bring that voice in even as early as the planning, everything's on paper, just trying to prove out, [to ask if] this is going make sense as a developer."
Gammill suggests that all of Microsoft's first party teams have been involved in the process of creating the Scorpio hardware. "[Turn 10] push us one way, 343 will push us a different way, the Coalition will push us a different way," he says. "Rare - they're pushing us a ton around Beam and some of the stuff that we may want to do there, so it's just great because we a super-strong relationship across the board with first party and it really benefits the platform, which ends up benefiting the customer."
Theoretical data and projections became very real with the arrival of prototype Scorpio hardware at the beginning of the year. In mid-January, Matt Lee from the Xbox Direct3D team arrived at the Turn 10 studio with a bunch of early components, eager to get Project Scorpio up and running with actual code.
"They wanted to get an early native title, one that was actually built for Scorpio," recalls Tector. "There'd been a ton of compatibility testing with previous titles but they really wanted to prove out one that was built against the XDK - and we could then do performance evaluation on it, running natively on the machine instead of through emulators and such."
Prototype Scorpio hardware at this point didn't really resemble anything that could recognisably pass as a console unit.
"He had a cardboard box full of parts and he starts getting them out on the desk and it's all loose motherboards, nothing's fitting together, it's debug connectors and it doesn't even have all the proper cooling, so there's just this huge PC fan - one of the bigger ones - and it's just sitting on the desk blowing across the top of it," laughs Tector. "It was comedy because it would start vibrating as the fan was running and it would slowly turn away and then it would point away from the memory and the whole system would shut down. We finally taped it down, that was our debugging rocket science! But all that, it was fun, really geek fun."
The process of getting ForzaTech up and running on the makeshift Project Scorpio hardware took just two days, and according to Chris Tector, the majority of that time was adapting the codebase from an earlier XDK [the PC-based development environment] that dated to just after Forza Motorsport 6's ship-date.
"There was one issue we had to deal with around memory alignment, then we were up and running on Scorpio. It was that fast. We were floored," he says. "We had this special room for [Matt], it was all locked away so he could be here for a really long time. It was two days and [on] the third day we were just playing around with all the options and re-doing all those stress tests and saying 'what if we turn on this MSAA or what if we turn on that'."
At this point I stop Tector mid-flow to clarify something that seemed to beggar belief. Was he really suggesting that the ForzaTech demo was fully performant straight away on this ramshackle collection of parts?
"From day one, yeah. We were floored... I mean, other launches haven't always gone that way, you're always fighting up until the final months... whereas now, from that second day we were able to say, 'well what are we going to do with all of this?'"
At this point, I'm ushered over to a work terminal with two displays. On the left, the PIX profiling software is running. On the right, the ForzaTech demo is running beautifully on a 55-inch LG OLED B6 display.
"This is us. This is ForzaTech running 60 frames a second, 4K," Tector says proudly. "We're still running with settings that we would have used in Forza 6. Since it's Xbox, we're using EQAA, so it's like a 4:2 EQAA. That's the actual GP utilisation so we're only using 60 per cent of the compute to get to this. Importantly, I know I've just said it's like a Forza 6 set-up, but this is also including 4K content, so all of our build system - we've got authored assets for this set of the models, cars, tracks everything. We pushed it through and made sure the 4K textures were flowing through."
The surfeit of memory bandwidth in Project Scorpio ensures that the additional overhead of these assets only hits performance by one per cent. The demo itself consists of a full complement of cars stacked up in two columns, moving like a train around the Nurburgring GP circuit.
"This is a stress case. Like I said, we do a ton of work to prove out the performance of any of our titles," explains Tector. "What we're doing is we're letting the cars drive. The full AI is running, the full physics is running for them and then we're snapping them back into this fixed grid... And then we let the AI player drive around and this is one of the cases we use to enforce the [rendering] budgets. We do a validation pass since all the cars or all the tracks or whichever components are running within budget. That's one of the biggest reasons we can hit those smooth frame-rates. We get super-stringent about how we follow those."
The Forza engine unfailingly hands in a new frame every 1/60th of a second - what Chris is saying is that each individual component has a set budget that it cannot exceed - but it can obviously come in under budget - for example, when fewer cars are on-screen. GPU load varies as the demo progresses; it seems to hit a maximum of around 70 per cent, but most of the time it's in the mid-sixties, sometimes lower.
"It'll vary around. Largely it scales relative," says Tector. "Here it's in the 50s because some of the car grid has gone off, and now we're back. And then it depends... are we on an outer section of the track and so there less for it to see? Whereas when we're at the start in the heavier sections, you've actually got more of the models in view."
The full complement of cars remaining on-screen for the majority of the duration ensures that this is a stringent test of the hardware, but maybe the Nurburgring GP course isn't the most demanding circuit available?
"They all adhere to a track budget," replies Tector, suggesting that the choice of course would have no impact on the result. "The way we do our budgeting is like at a sub-frame level. It's not did you hit 60? It's did the environment hit this number of milliseconds budget, so that normalises everything and lets us manage it as a whole process across the hundreds of cars and all of the tracks that we have. Otherwise we wouldn't be able to manage it."
The presentation is crisp and clear, the Scorpio test kit sits next to the TV - there's no doubt whatsoever that this is indeed Scorpio hardware running the Forza Engine at a locked 4K60 with plenty of overhead remaining, and GPU utilisation is definitely staying within a set range as the train of cars careens around the track. The result is impressive, but without seeing an Xbox One running the same demo as a reference point, the question is how much scalability we actually have with the jump from 1080p to 4K.
"With this scene, it's good that you point that out because we don't have a few of the components that we have within the budget, like particles. I'd say it would probably be 14ms but with this type of a grid it might be closer to 15ms," Tector replies - effectively suggesting that GPU utilisation in the mid-60s on Project Scorpio hardware would translate to 84 to 90 per cent running the same content at 1080p on Xbox One.
Tector moves to the left display on the bench, running PIX. He scrolls through GPU captures where the waves of work roll out. The tool can show what shaders are running, what the bottlenecks are, and he can dig deeper, all the way down to the individual GPU draw call. There's a similar level of granularity on the CPU side too. It allows for fine-grain optimisation that simply isn't possible on PC.
"When we did Apex and we were first going on the PC coming from Xbox development, we had these well tuned performance tools, going back to PC development where we didn't have them was painful," Tector recalls. "For us to have this from day one, it's been so powerful because we just get insights instantly and we can start comparing and saying 'well I know it took this many microseconds to render this driver model on Xbox One. Why is it taking this many microseconds or nanoseconds on Scorpio?'"
This immediate level of information on a brand new console is hugely beneficial to developers. Microsoft says that Project Scorpio sits within the Xbox One family of products and it's clear that this goes all the way back to the developer level - the mature tools used to optimise for the three-year-old console work just as well with its upcoming 4K-orientated companion.
"Since those first two days we've actually shaved out a couple of milliseconds because we need more time, we want to bring more features into the game and importantly we want to bring them into the game in both versions," Tector reveals - and it'll be interesting to see what they do with that additional GPU time in the next Forza. "Having that level of detail there lets us pull off optimisations that only a first party is going to do and usually they only do them for the one console they're working on. And now the two of them are so similar and so compatible, we can apply optimisations over here that are going to benefit Xbox One. We can actually bring features to both versions we might have not have been able to."
Clearly, there's still a large amount of GPU overhead left over on Scorpio, so where can the engine be taken from there? Turn 10 is still working on this, but the quality settings from Forza Motorsport 6 Apex on PC can be fully invoked on Scorpio. Turn 10 literally ramped up everything to ultra and it just worked, with the game retaining a 4K60 performance level. As well as validating the capabilities of the new console, we also get an insight into how some ultra-level settings may actually amount to a rampant misuse of PC GPUs' capabilities.
"The crazy story here is that we've gone over our PC ultra settings and for everything that's GPU-related, we've been able to max it - and that's what we're running at, 88 per cent," says Tector, pointing to the utilisation data at the top of the screen. Right beneath it is the anti-aliasing setting - 4x, or rather 8:4x using the Radeon EQAA hardware AA.
"This is rendering the player LOD for every car, so you won't see a single LOD pop. [It's the] top-level model you'd see in race, one below what you see in Autovista, the model you usually only see for the player. And then we balance out LODs across the scene. It's a disgusting abuse of GPU power is what it is, right?"
With the equivalent of PC's ultra settings invoked, shadow quality and corresponding resolution are ramped up, more subtle areas of the lighting are enhanced, foliage density increases and the number of taps used for motion blur quality doubles. Based on the ForzaTech screenshot Microsoft provided for our Scorpio coverage, essentially the same level of geometry used for the car nearest to the camera is being used on the vehicle furthest away. Taking the shot into Photoshop, it's occupying approximately 1500 pixels out of 8.3 million - a disgusting abuse of GPU power indeed.
"It's horrible but we can do it and we still have the power left over," Tector concludes.
The extent to which the comparison is valid is highly debatable, but we tried to replicate the ForzaTech content on the PC version based on this ultra-level stress test. It's impossible to keep all cars snapped into place as per the Scorpio demo, but the evidence seems to suggest we're looking at performance in the same ballpark as an Nvidia GTX 1070-class GPU here and even that can drop frames at ultra settings when wet weather hits. Turn 10's stringent budgeting would ensure that this would never happen in a console release. But the point isn't really about platform comparisons at this very early stage, it's more about how Turn 10 plans to use that power to get more visual return for the console player as opposed to just maxing out quality settings that offer a drastically diminishing visual returns.
"The awesome part about the whole story [is] that we can spend all this time heading into the future," says Tector. "Instead of saying how are we going to wrestle to get the performance on this, we're actually saying we can make this quality trade-off or this quality trade-off and spend that time iterating heading towards much better image quality - so instead of stressing about getting to a final resolution for titles or a final frame-rate, we can really drive it all into quality."
Unfortunately, the Microsoft team won't show me everything they've got up and running on now-final development hardware - E3 will be the venue for that - but they're keen to point out that despite Turn 10 running one of the most optimal game engines in the business, others are getting similar results. I'm also told that key developers have had access to hardware on site at the Microsoft campus.
"Chris's experience is not atypical of what we've seen from partners we've had in - varying, different degrees of results but all positive. It's been great," says Kevin Gammill.
"I think this is the earliest in terms of time to ship. As Chris was alluding to, it's usually right down to the wire," adds Albert Penello. "And now we've got fully performant boxes going out shortly... We're seeing exceptionally fast 'getting up and running' times with developers, so we're pretty bullish. I like the direction we're heading, where we're hearing a lot of time spent on quality, not a lot of time spent on performance tuning or engines or the technical development side."
I went into the Xbox visit looking for validation beyond the specs, that the Scorpio hardware could deliver on the claims made for it. It speaks to the confidence of the Xbox team that they chose to showcase the power of the new console by wheeling out a three-month-old demo (!).
As Chris Tector points out, the Forza engine has evolved since then with further optimisation allowing for the addition of new features for the next series release. And on top of that, Turn 10 still has that big slice of GPU time remaining to beef up Scorpio visuals still further. It'll be fascinating to see how those resources are deployed in the final game.
It's only one game engine of course - and one of the best in business, no less - with all the benefits of a first party focus. In terms of a better look at how Scorpio will handle a breadth of software, we'll need to see more games - third party titles in particular. But at this very early stage, I asked for an authentic real-time demonstration that shows what the Scorpio hardware can potentially deliver and Microsoft didn't falter. And going forward, it sets up an extraordinarily high level of expectation for E3 - just two months ahead of us. The hardware credentials are clearly impressive, but as always, it's the games that count.
We learned about Forza Motorsport on Project Scorpio at an exclusive briefing at Turn 10 Studios Microsoft paid for travel and accommodation.