Switch's mobile mode analysed in unprecedented detail
And the hardware mod that makes it possible.
Last week, we took delivery of a new and very special Nintendo Switch unit, based on retail hardware but with a very special feature that'll radically improve our coverage. To cut a long story short, this new consoles offers us the ability to capture direct-feed video while playing in portable mode. On the face of it, it's a feature that doesn't offer much use to the average user - after all, you can dock the system and use HDMI to hook up the system to your HDTV. But for Digital Foundry, it's a game-changer - we can finally analyse Switch's portable configurations to see how they stack up against the docked experience, and based on our very first tests, some of the results are fascinating.
Let's quickly remind ourselves of how the Switch works in mobile mode. The ARM Cortex A57 CPU cluster runs at around 1GHz whether you're docked or undocked, but GPU clocks shift significantly - dropping from the 768MHz of the docked configuration to 307.2MHz or 384MHz when gaming on the go. Basically, developers have the choice of opting for more muscle at the expense of battery life, but almost every 3D-based Switch title has a series of compromises in place for handheld play. Previously, we could only look at still screenshots or low quality 30fps video via Switch OS (on supported titles only) but now, we can see everything.
The hardware mod itself is available to everyone, with units available to purchase from the creator, 'Katsukity'. You can either buy a completely new unit or else get your existing console modified. Long-term Digital Foundry users might remember when we reviewed a video output mod for PlayStation Vita - yes, it's the same guy, who makes a living by adding video capabilities to various handheld consoles, up to and including the Wii U's GamePad.
When fitted, the mod features a rear-mounted board encased in a plastic shell with a USB 3.0 connection on top - this attaches to your PC and special viewer software is used to display the image on your monitor. It's a little unwieldy and HDMI would have been a lot simpler to process, but it does the job with no frame loss (though we recommend attaching to USB 3.0 ports on the rear of your PC - front-facing ports tend to be based on a hub, which can impact data flow). The dock itself is also sliced up to better house the modded Switch console. Overall, the end result does look a little a messy but crucially, it works.
The viewer includes plenty of options allowing you to adjust how the image is scaled and the size of the backbuffer you wish to use for capture. It's amazing just how sharp the image can be - 720p scaled to 1440p or 4K appears gloriously crisp. Even the Switch's own UI benefits from this. You can use nearest neighbour integer scaling here for a look that I prefer to the standard 1080p output you get from the dock.
But really, it's all about the games and the new insights we can glean from them. The first title I loaded up was The Legend of Zelda: Breath of the Wild. It's one of the best games on the system and one that makes great use of the hardware. It also demonstrates the benefits of a smaller screen - in portable mode, Zelda uses an adaptive resolution that bounces between a full 1280x720 and 648p but while playing, this is difficult to spot. Sure, you might notice a slight loss in clarity but avoiding frame-rate drops is far more critical. With the mod in place and the display output blown up, it's easy to see when the game shifts between the two different resolutions. The adaptive resolution kicks in far more frequently than initially thought and it demonstrates how much leeway working with a mobile screen offers developers using various sleight of hand tricks in improving performance for a better handheld experience.
At launch, Zelda had the interesting trait of offering smoother performance when running in mobile mode compared to the docked experience, but after further patching, the game seems to run at its target 30fps in both configurations now (at least in the early stages - Switch's lack of cloud saving, or backing up of save game data, limited the extent of our testing). However, it does seem that performance can indeed improve in mobile mode on select titles - including the nigh-on miraculous port of Doom 2016. We re-examined this title earlier in the year with the latest patch installed and found that in docked mode, the average resolution had increased producing a slightly sharper overall game experience. However, with the new capture kit, we can confirm that key hotspots in the game do indeed run smoother in handheld mode.
Of course, with its reduced clocks, sacrifices are necessary and that comes in the form of reduced resolution - while docked mode spends a lot of time between 648p and 720p the portable mode regularly drops to around 432p or so during gameplay. While the smaller screen helps alleviate this somewhat, 432p is undeniably low, so the game appears fairly blurry. However, the trade is clear - by widening the resolution window, Doom can run smoother in mobile play than running docked.
The obvious follow-up test would be Panic Button's excellent Wolfenstein 2 conversion. There are no performance increases in mobile mode this time - it does run less smoothly, that's clear - but remarkably there are occasional areas where resolution is actually higher when running in portable mode. Docked mode can fall back on sub-540p resolutions to help mitigate performance dips but since portable mode is already at this level most of the time, there's nowhere else to go, so performance suffers. And it does highlight that while the smaller screen can hide a lot of visual compromises (the removal of ambient occlusion in portable mode essentially goes by unnoticed), there are limits. While 648p was barely perceptible in Zelda, the drops to 432p and 360p are obvious during handheld play.
But it's not just third-party titles that employ specialist tricks and techniques to make mobile play possible. As noted in our original Super Mario Odyssey analysis, I discovered that Nintendo is jittering two 640x720 images together all the time in mobile mode. So, in a completely static scene, we get the illusion of a full 720p image but in reality, the resolution per frame is lower. This is an effective strategy when played on the Switch's mobile screen but blowing up the image using the capture kit demonstrates why we haven't seen anything quite like this in docked mode - it doesn't work well. It is, however, a great example of how developers are using portable-specific tricks to help preserve the full-fat console experience on the go. You can spot this during gameplay if you look closely at the screen, but it doesn't stick out immediately. Instead, it's the way in which Nintendo has managed to preserve the experience in mobile form that really matters most.
Of course, not every developer is this successful when it comes to finding a good solution for portable play. Xenoblade Chronicles 2 is an example of portable image quality gone wrong. As we discussed in our original analysis, it uses a strong sharpening filter that produces noticeable, often severe edge ringing artefacts. It looks over sharpened at all times and while bearable in docked mode, portable gameplay is severely hampered by it. On top of that, the massive drop in resolution is also accompanied by a small drop in performance too. Xenoblade 2 is one of a few games that suffers a lot in portable mode: performance is worse on average and image quality is abysmal. It may not drop as low as Wolfenstein 2 in terms of pure pixel count, but the image treatment used ensures that it looks much worse. Even on a small screen, the issues are obvious.
In our tests so far, it's all been about compromising on resolution when moving to mobile play, but the Bayonetta collection is interesting in that both titles run at 720p in both docked and portable modes, and the chief difference concerns performance instead, with docked play aiming for as close to a locked 60fps as possible, while frame-rates in mobile mode match and inch ahead of the Wii U originals. Mario Kart 8 is another title that offers a Wii U-level experience on the go: image quality is great in portable mode with a full 720p displayed throughout our tests. It turns out that the Wii U actually employs dynamic resolution scaling on its busiest scenes and while this may be employed on Switch too, all of our tests indicated that even when running in handheld mode, resolution stays solid at the screen's native resolution.
There's a lot more we want to do with this mod as beyond Bayonetta, there are a couple of titles that actually run at 1080p on Switch in both mobile and docked modes (Payday 2, assuming it hasn't been patched, and in addition to that, Hyrule Warriors) so just for starters, there's the potential to actually benchmark Switch in mobile vs docked configurations, to see how much that extra docked GPU power translates into raw performance, but I decided to end these initial tests ended with a look at a couple of very popular multiplayer games.
Paladins from Hirez Studios looks glorious in portable mode and blowing it up to full-screen on a PC monitor doesn't detract too much from the presentation and I'm impressed at the fluidity of the game and the resolution of the experience. When respawning or while engaged in larger battles, the resolution can drop pretty low - all the way down to 378p. The thing is that this doesn't really seem to happen all that often based on my experience and it's an impressive showing for an Unreal Engine 3 game, bearing in mind that the technology has no official support from Epic itself.
I also couldn't resist trying Epic's own Fortnite - it's a wide-open Unreal Engine 4 title that fares pretty well on Switch. Like Paladins, the resolution can slip but I found image quality to be a step back in some ways. The aggressive temporal AA technique can produce some ugly moments, but again, they mostly pass unnoticed on the smaller screen. It's also interesting to compare against the iOS version: phones like iPhone X offer a more powerful GPU than the Switch, which results in a higher overall resolution across the run of play. However, for its part, Switch's low-level API access allows for more features and the inclusion of techniques like dynamic resolution scaling and temporal anti-aliasing. Once the Android version appears, it'll be fascinating to do a full mobile head-to-head to see just how well Nintendo's handheld compares against the best mobile GPUs on the market.
Overall, getting to grips with the new hardware-modded Switch has been highly educational. It's confirmed a lot of our initial thoughts about how the mobile modes in various titles work, while providing a lot of additional insights about the techniques used to get some high-end experiences working on a handheld. And by extension, it's also given us an better appreciation of the development effort that goes into these games on what is a fascinating and unique console platform.