Digital Foundry vs Xbox One streaming on Windows 10
The good, the bad and the ugly.
With the launch of Windows 10, Microsoft finally has an answer to Valve, Sony, and Nintendo's own off-TV streaming options - Xbox One Game Streaming. Utilising the Xbox app included with the new OS, Xbox One owners can now play console games on any PC running Windows 10, via their local network. On paper, this seems like an excellent addition to the OS that could enable players to keep gaming even when their primary television is tied up. Unfortunately, a number of issues prevent this feature from truly shining in comparison to its competition.
The streaming is simple enough to set up - just fire up the new Xbox app and look for the 'game streaming' option on the startup screen. The first time you attempt to stream, the app searches your local network for any available Xbox One consoles which you can promptly select and begin using. A streaming test is available but offers little information outside of 'yes, it will work' or 'no, it won't'. In addition, the option to add your console via IP address exists to support alternative network configurations. In our case, both our primary PC and Xbox One console were connected via Ethernet to the same router but we also sampled the feature over WiFi.
The app itself is easy to use and supports a number of control and accessory combinations without any additional configuration required. You can either leave your Xbox One pad paired to the console itself or directly connect an Xbox One or Xbox 360 pad to your PC instead. When connected to your PC, it's even possible to use the microphone input along with other similar accessories. The app enables users to monitor bandwidth statistics but currently only 'total bandwidth' is useable - other stats, such as 'end-to-end latency' and 'packets lost' simply display blank containers when engaged.
Perhaps the most impressive element is the fast input response. Without the ability to monitor latency via the app, we resorted to running our own test by filming our monitor at 120fps and analysing the difference in controller response between a direct connection and Windows 10 streaming. Knowing that the display used in this test operates with 23ms of input latency, we were able to determine that the streaming app incurs an additional six frames of lag. With the footage being recorded at 120fps, we're really looking at an additional three frames here leaving us with around 50ms of latency. This is indeed quite responsive but, unfortunately, also somewhat variable.
The real issue with the streaming feature lies with its performance - we were simply unable to attain a consistently smooth update: the higher quality settings attempt to operate at a full 60 frames per second but regularly fall short. The high setting requires somewhere around 13-14mbps while medium is closer to 6mbps. Throughout most of our testing, the play experience simply wasn't as smooth as it should be. Knowing that Steam in-home streaming supports upwards of 30mbps and operates flawlessly on the same network, it's safe to say that there are still issues to resolve here.
What's frustrating here is that we were actually able to reach a completely stable 60fps during gameplay in a select few instances. Monitoring in-app bandwidth usage and general network bandwidth availability failed to uncover any bottlenecks with stats remaining consistent in all scenarios. You can see this with the Forza 5 footage included in our performance analysis - the first 20 seconds are very fluid but frame-rate quickly degrades. As far as we can tell, bandwidth is not the limiting factor here.
We also tested streaming via WiFi using a laptop running Windows 10. In our case, performance was nigh identical over WiFi versus a wired connection. All of the same problems we encountered on the desktop present themselves on our laptop too. Of course, using a smaller, lower resolution (1366x768) screen does at least produce a more attractive image - the stream effectively downsampled here, and lowering the hit to quality from the compressed video stream.
Which brings us to image quality in general. Three quality settings are available by default, but even the highest setting fails to provide great image quality. The results appear much softer than expected with a lower overall resolution in an effort to save on bandwidth, no doubt. The selected video compression utilises 4:2:0 chroma subsampling which has an impact on colour resolution and vibrancy while macroblocking and other compression artefacts further compromise the image. How noticeable these issues are varies between users and monitors. Streaming games to a Surface tablet with its rather small screen helps conceal some of these issues that are supremely noticeable on a large 32-inch PC monitor.
Of course, this using the readily selectable default presets. Thanks to the detective skills of a Redditor, an additional very high option has been uncovered in the Xbox app. So how do you unlock the hidden quality preset? Well, assuming Windows 10 is installed on the default C:\ drive, simply navigate to C:\Users\%USERNAME%\AppData\Local\Packages\ and then locate the folder starting with "Microsoft.XboxApp". From there, open the 'userconsoledata' file using the Windows Notepad app, and where the 'IsInternalPreview' variable is set to 'false', adjust it to 'true'. Save and close the file and the very high preset is now available within Windows 10's Xbox app when you fire it up.
The primary improvement offered by the hidden setting seems to be an increase in resolution - now full 1080p - and an associated bump in useable bandwidth to 20mbps. Unfortunately, the good news ends there as compression artefacts remain ever-present and performance is still inconsistent. This is far and away the best option and does provide the best overall experience, but it certainly doesn't solve the most significant issues. Clearly, it still lags some way behind Valve's implementation. Keep in mind that this is not officially supported by Microsoft and, while it seems harmless, there may still be unknown issues associated with this change.
The app defaults to full-screen output with an optional windowed mode available. Some users are reporting freezing issues with the full-screen mode but that wasn't a problem in our case and we were able to run in both configurations with no issues. It should be noted that the full-screen mode simply scales to the resolution of your display, which means 1440p in our case. It would be nice to enable a smaller, 1:1 pixel mode or at least force the windowed mode to operate in this way.
We also ran into various other issues outside of the actual streaming process. While the app is easy to use, we encountered a number of stability issues that required additional steps to resolve. In one instance, due to an issue on the console side, we opted to completely reboot the Xbox One. Once finished, we attempted to connect via the Xbox app only to be told that the setting enabling streaming was disabled on the Xbox One itself. This was not the case.
So we booted up the console on our TV, toggled the setting, and tried again. This time it worked. Unfortunately, the Xbox was set to output at 7.1 via the receiver while it requires stereo mode within the streaming environment for audio playback. Once we powered off the receiver and TV, we were no longer able to hear any sound from the console in Windows and switching to 'uncompressed stereo' on the Xbox itself produced an error. A simple power cycle solved the issue but it's something that has popped up a couple of times now.
This, coupled with the highly inconsistent performance, leads to a very uneven experience. Compared to Steam in-home streaming, the results here are clearly disappointing. - even Sony's Remote Play feature and PS Now offer more consistent results. It works but it just doesn't work as well as we'd like. Input response is generally fast, which is very important, but the frame-rate just isn't where it needs to be. 60fps titles regularly dip under the desired level of performance while 30fps games exhibit frame pacing issues during streaming. In all cases, we were never able to achieve a stable, consistent experience for any length of time.
Ultimately, it's difficult to pass judgment on this feature. It's still early days for Windows 10 and it's entirely possible that we'll see improvements made to the streaming functionality over time. It's also difficult to judge a networked solution such as this using a small sample size. We tested game streaming on two separate networks, one of which is using a relatively high-end router, and the results were similar. Yet, there is video evidence out there of other users enjoying relatively smooth experiences in comparison. Our experience wasn't great but that doesn't mean others won't fare better. Seeing that this is a free feature included with Windows 10 it certainly doesn't hurt to try it - just don't expect a class-leading experience for now. The few moments we experienced where Xbox One streaming worked flawlessly proved that the tech is capable of delivering - we just hope that Microsoft can iron out the issues and get this feature working to the full extent of its potential.