Google Stadia hands-on: streaming analysis and controller impressions

We've talked elsewhere in our Google Stadia coverage about the fact that it's not a console, it's what Google calls a 'cloud native' system that leverages tight integration of gaming components in the datacentre to provide - in theory, at least - a system capable of delivering genuinely different gaming experiences, while providing a generational leap in processing power over today's consoles. That said, we should expect to see ports of multi-platform games and Google itself has already demonstrated one of them - Assassin's Creed Odyssey, delivered to the US public at the tail-end of last year for extensive beta testing. We had the chance to go hands on with the game again, running on the latest version of the streamer, and with access to Google's own controller hardware.

It was also a chance to get some answers to some outstanding questions we had from the Project Stream beta phase. Was it actually running on Google's new Linux/Vulkan-based platform, or was it merely an infrastructure test based on PC code, or something else entirely?

"[It's] a complete port," Google's Phil Harrison told me. "[Ubisoft] built the game completely for Stadia and they're actually doing a talk at GDC about how they got their game up and running."

In our earlier analysis, our impression was of a game that very, very closely mirrored the PC version running at 1080p resolution, with some elements upgraded beyond the console quality threshold defined by Xbox One X.

"Correct, but they started from their main line on the consoles, it's not that they took the PC version and ported," Stadia VP Majd Bakar explains. "You can see that as the UI changes according to the controller you connect. I wouldn't call it a console port, I'd recommend going to the talk. It's going to be run jointly between Google and the team that did the work for Project Stream and they will talk about how they did it and the work stream they followed."

We'll report back on this should any further details emerge on how AC Odyssey came to Stadia, but in the meantime, we got the chance to re-test the game, safe in the knowledge that this is indeed running on actual Stadia hardware, apparently with a more recent version of the backend streaming technology in place.

We're still not at the production-level quality, though. The demo we played retained the 30fps performance profile we saw in the Project Stream demo, and the video quality itself was still at 1080p - Google is promising 4K and HDR for the final release. On top of that, the system itself we saw lacked any of the YouTube integration or other cloud features. Furthermore, we're certainly not in controlled conditions for the tests that are to follow, but we can provide new analysis and data points for the system's performance and get hands-on with some new hardware - in fact, the only bespoke device Google will ship when Stadia launches.

Google Stadia Specs Analysis + Exclusive Performance Testing

The controller: Google's only hardware offering

We've established that Google isn't making a console, instead leveraging the ubiquity of the Chrome browser and its Android OS to deliver streaming clients for computers and phones, while Chromecast delivers a gaming experience for any HDMI display. What's missing is a controller. For computer owners, any USB HID device - like a Dual Shock 4 or Xbox joypad - will do just fine, but those using Chromecast lack any kind of input device. That's where Google's controller, pictured below, comes in.

Three aspects set this device apart from other pads. Firstly, although you can hook up a USB-C cable and use it on your computer (it worked just fine on my Mac), it's primarily WiFi-based. The Google controller is actually a client device in its own right, meaning it has a direct connection to the cloud server and doesn't need to go through your screen, phone or computer. It also makes it easier to swap your game instance from screen to screen - by default, the server instance persists and the same game is playable if you decide to switch from, say, living room display to your laptop. The controller also has a share button (Stadia automatically saves off a 4K60 HDR stream of your gaming) and in addition to that, there's an option to invoke Google Assistant for voice control of what would usually be traditional UI features (though there is a UI too). So, yes, the Google pad has an in-built mic.

Having used the controller, I'd say that what it lacks in distinctive looks, it makes up for in terms of response. The feel of the controller, right down to the plastics, is quite Xbox-like and if you're used to that pad, you'll easily be at home with this one.

Google Stadia: latency analysis - re-testing Assassin's Creed Odyssey

But what about the games? Assassin's Creed Odyssey took point in the Project Stream demo and it's still Google's 'hero' game. In our prior testing of Assassin's Creed Odyssey on Project Stream, tested on a home internet connection, we pegged button-to-pixel latency for the system at 179ms. The game - which we now know to be a full port of the main line console version of the game - operates at 1080p at 30fps, the half-refresh frame-rate adding to the overall lag. Our measurement here was using a LAN connection to the router, with 200mbps of bandwidth.

I had the opportunity to re-test the system using a later version of the streamer, playing the game on a Google Pixelbook, running ChromeOS and connected via WiFi. The chances are that the internet connection here is even more capable than the 200mbps connection we used previously, but despite the fact that we used WiFi (which adds anything up to 10ms of lag in our previous tests, though router and device configuration could change that) and that we don't know the display lag of the Pixelbook, our most consistent button-to-pixel result is 166ms. Google also offers developers a 'worst case scenario' mode to allow for testing in adverse conditions, which simulates a poor, noisy 15mbps DSL connection.

As the tests are comparing data that's not fully like for like, I wanted to demonstrate the range of latencies you could encounter, so on the second row of the table, I've added back in the complete end-to-end totals for the local experience. The LG C8 we used adds 21ms of lag in Game Mode.

* The Stadia tests were carried out on a Google connection vs our Stream tests 'in the wild' on a 200mbps connection. However, the Stadia tests include display latency on a Pixelbook (which we cannot measure) and were also carried out via WiFi networking. Both of these factors will add additional latency, while our Stream demo tests were carried out via a LAN hook-up to the router.

** The '15mbps' mode is a simulation mode offered to developers, designed to mimic response on unstable connections. Image streaming takes a hit also, in terms of both resolution (1080p becomes 720p) and quality.

*** This comparison stacks up overall measured latency end-to-end, with the local machines running on an LG C8 OLED display with game mode enabled, adding 21ms of lag. The Pixelbook display's latency is unknown.

Of course, this is not controlled conditions and Google's connection possibly dwarves the 200mbps link we initially used in our Project Stream tests. But the inescapable conclusion seems to be that gap has closed since we last tested - a full frame's worth of latency has been saved, and almost certainly more than that, as we cannot factor in the Pixelbook's WiFi and display lag, two factors we eliminated from our Project Stream tests easily enough. Assassin's Creed Odyssey is perhaps not the best example for testing, as it's natively a fairly laggy game, so it will be interesting to test faster 60fps titles.

In terms of controller response though, AC Odyssey on Stadia feels very close to a local experience - timing-sensitive moves like parrying are achieved with no problem whatsoever - and that brings us on to another latency element. We tested using the Pixelbook's keyboard, while Google says that its controller's direct WiFi link to the server delivers further latency advantages.

One thing that surprises me now that we know of Stadia's spec is the decision to cap at 30 frames per second. With a 10.7TF graphics core and fast, server-class CPU, we would expect the Stadia port of Assassin's Creed Odyssey to run at 60fps. Based on the results we're seeing here running the PC version at 60fps, latency should drop by 33ms. The Xbox One X version is already bafflingly high in latency terms, to the point where Stadia can match it. Based on the PC results, running at full frame-rate would actually see the streaming version surpass the Xbox game.

Google Stadia: image quality testing

If you have a very wide internet connection, Stadia supports 4K video at 60 frames per second with HDR support. However, based on conversations with Google, we expect that 1080p streaming is more likely to be the norm on connections with around 25mbps of bandwidth. Stadia will go lower, dropping down to 720p60 with connections in the region of 15mbps - but having seen it in action with an artificial cap in place, it's clear that fast action causes obvious artefacts on a large, living room display. Google itself considers this to be a 'worse case scenario'. The experience would likely hold up much better on a smaller screen, however.

In putting together this image quality comparison, you can see how the latest version of Google's streamer copes with the same scenes we tested previously in Assassin's Creed Odyssey. By and large, the quality delivered seems to be the same - with perhaps some small improvements in more taxing scenes. Assassin's Creed Odyssey looks fine overall and the hit to quality doesn't unduly impact the experience.

Stadia is perfectly playable and presentable here, but it's clear that there is a noticeable visual hit when the encoder - which is a bespoke Google creation, and not a part of the AMD GPU - is presented with more a more detail-rich, fast-moving scene process. So by extension, we have two potential concerns here - firstly, despite the Stadia spec having more than enough power to run the game at 60 frames per second (based on the requirements of the PC version), the demo runs at 30fps. Every other frame is identical then, making life easier for the encoder - though this does mean that artefacts persist on-screen for twice as long, making them more noticeable.

Launch comparison tool

Secondly, there are far more visually complex games out there that would be even more of a challenge to the encoder. It may well be the case that developers may wish to ramp up the post-effects pipeline with effects such as motion blur, which can look visually pleasing and will pose far fewer issues for the streamer to deal with. Assassin's Creed generally could certainly benefit from an optional motion blur effect!

Google Stadia: first impressions

There are a number of takeaways from our Stadia hands-on, both positive and negative. First of all, we need to stress once more that the tests we carried out were not in controlled conditions, but even so, controller response was the best I've experienced from a cloud system to date. Having spent some time with Assassin's Creed Odyssey on Xbox, the feel of the game is very similar indeed to what Ubisoft and Google have delivered with their port. Latency is higher than PC, especially so when played at 60fps, but I do have to wonder whether it's the actual latency that is important or the consistency, bearing in mind the wide delta in display lag.

Also consider the input lag numbers we reported for the key FPS titles back in the day. Doom 2016 and COD Infinite Warfare are both 60Hz shooters, but we noted a 47ms latency difference between them in COD's favour (39.3ms vs 86.8ms). To our knowledge, few people have any issue with Doom's controls. I do believe there's a personal threshold level for lag sensitivity, and it's particularly noticeable on 30fps games that can drop below their target frame-rate. My guess? If Stadia delivers consistency at home and retains the lag difference seen in these tests, most gamers will be fine with it.

Image quality is a slightly different issue, and I believe that Stadia's fortunes here depend on two factors: encoding quality and screen-size. Writ large on a living room display, fast action in Assassin's Creed Odyssey does tend to break up - the mitigating factor here being that it's actually a slower-paced game, so by and large, the game runs just fine and looks good. Screen-size definitely has an impact, in several respects. First of all, playing on the PixelBook, the reduction in real estate is such that image quality holds up a lot better - and the improvement is even more notable on a smartphone screen, which I also tested with a Razer Raiju Mobile. However, what quickly becomes evident is that console and PC games simply aren't designed for small displays, resulting in profound legibility issues on HUD elements - I think this is an aspect of gameplay that major streaming services haven't really considered yet, but should hopefully be built into newer games. Gaming on a laptop though? No problem - AC Odyssey looked great on my 13-inch MacBook Pro.

Overall, while my tests with Stadia were somewhat limited (the encoder in particular needs stress-testing across much more content), there are many reasons to be optimistic about Google's offering: instant access to your gaming library across a range of different devices, reasonable image quality and decent response, along with improved loading times. There does seem to have been significant improvement since we tested Project Stream a few months back, but more than that, the system will scale in both hardware specs and performance over the years as infrastructure and Google's internal technology improves. Stadia will launch in North America, Canada, Western Europe and the UK by the end of 2019 - and I'm really looking forward to really putting it through its paces.