Why are some PS4 Pro titles running slower than base hardware?
UPDATE: Watch Dogs 2 also runs more smoothly on the standard PS4.
UPDATE 15/11/16 9:20am: While we've had some excellent PS4 Pro results recently from the likes of Call of Duty: Infinite Warfare, Battlefield 1 and FIFA 17, our latest testing with Watch Dogs 2 reveals some concerning results.
For the majority of gameplay, the game hits its 30fps cap and delivers a crisp 1800p image, achieved via checkerboard upscaling and delivering excellent image quality. However, what's clear is that stability in performance is an issue, with the game tearing and dropping frames sporadically during city traversal, while the base PS4 game runs without issue. Police pursuits ramp up transparent alpha effects which clearly exacerbate the problem - surrounded by squad cars with a helicopter search light illuminating the scene, Pro drops frames badly and introduces screen-tear when, once again, the standard PS4 retains its 30fps cap.
Again, the concept of the top-tier console producing sub-par performance isn't really acceptable and it's dismaying to see results like this. The bottom line is that Pro users with 1080p screens may well get a boost to image quality via super-sampling, but reduced performance under any circumstances can only call into question the whole point of a more powerful console.
Original story: PlayStation 4 Pro testing continues at Digital Foundry and while impressions remain positive for what this system is capable of - especially when paired with a 4K display - there's an aspect of platform performance affecting some games that we find concerning. Several titles are running at lower performance levels than the base PlayStation 4 versions, presumably owing to higher resolution support. As we understand it, this constitutes a fail according to Sony's technical requirements for developers, so it's fair to say that we were not expecting to see this issue manifest in completed launch software.
It's certainly not a universal issue that applies to all software (many of the upgrades we've seen so far have been tremendous) and the extent of the issue varies not just from title to title but also from scene to scene in affected releases. However, the bottom line is this: for games like Skyrim, The Last of Us and Mantis Burn Racing, gameplay will actually be smoother overall on base PlayStation 4 hardware [UPDATE 19/11/16 6:24pm: Mantis Burn Racing has had a title update and now runs very smoothly]. Deus Ex: Mankind Divided is a little more complicated: we find that some scenes play at a noticeably lower frame-rate, while others hand in more consistent performance overall.
For 1080p support, the Pro down-samples the 4K mode for all of these titles, offering enhanced image quality via super-sampling. However, the areas where performance is degraded remain impacted when running on a full HD display - meaning that there's nothing that Pro owners can do to get a smoother experience overall on these titles. In fact, the base PlayStation 4 will run these releases at a higher performance level overall at the expense of some image quality.
Probably the most surprising example of this is The Last of Us Remastered, a Naughty Dog production. The team has a reputation not just as one of the most cutting-edge developers in the industry, but also for their superb optimisation skills. Generally speaking, The Last of Us has been an absolute joy to revisit on PS4 Pro. You get a 3200x1800 resolution in 60fps mode, rising to native 4K at 30fps. It's also one of the best HDR titles we've tried. So if you have a 4K display, losing 2-5fps in performance in the 60fps mode doesn't feel like a bad trade as such, but the point is that you cannot opt for enhanced performance instead.
However, if your Pro is connected to a typical 1080p screen, there's no HDR and the resolution boost is repurposed into enhanced anti-aliasing via down-sampling. At that point, the trade isn't so favourable - we want the game to play as smoothly as possible. And to be frank, this is PlayStation 4 Pro - we shouldn't need to be discussing compromises at all, and the experience should be protected via Sony's technical requirements that apply to all game-makers. All Pro software should be running at the same frame-rate or better regardless of resolution increases or any other enhanced features. The Last of Us is the most obvious example, because for the most part, any perf impact on the base PS4 hardware is worse on the Pro. Meanwhile, in some scenarios, the standard game runs at a locked 60fps, while the Pro may drop frames.
The Last of Us stands out because the performance differential is fairly frequent. However, other titles only show minor fluctuations during general gameplay and for the most part, performance is mostly on par. For Skyrim for example, the general experience feels pretty much like-for-like - it's a game with a 30fps cap, which the Pro version adheres to, by and large. Enhancements include improved foliage draw distance, but the key boost is resolution - the Pro version runs at 4x the resolution of the standard PS4 version, offering full, native 4K. However, in scenes heavy on alpha transparency effects, frame-rate can drop where it doesn't on base hardware.
Super-sampling on PS4 Pro offers detail improvements, but the anti-aliasing boost is actually rather limited, as the game's base temporal AA solution at native 1080p resolution is actually very good. The presentation is fairly soft anyway, so again we have a 1080p/4K divide: on an ultra HD display, we'd rather play on Pro and enjoy a huge resolution boost all of the time and frame-rate dips only occasionally. However, at 1080p, we'd prefer more consistent performance throughout the game and so the base PS4 provides a smoother experience overall. But the point is that once again, we shouldn't be seeing these divides at all, on any title.
Deus Ex: Mankind Divided is an interesting game. Just testing the first mission, followed by an engine-driven cut-scene, throws up some interesting results. The higher resolution mode consists of a dynamic framebuffer system that scales all the way up to 4K, using the checkerboarding technique. The Prague cinematic is notable for its poor performance on base PS4 hardware, while the Pro actually improves on it in most scenarios - boosting frame-rates overall and eliminating tearing. However, the first mission runs at a close to a locked 30fps on base hardware and it doesn't on Pro. At worst there are 20fps dips - something that surely isn't acceptable for a Pro experience. The alpha-heavy conclusion to the first mission sends frame-rates plummeting on Pro while they're closer to a more solid 30fps on base hardware. It sticks out like a sore thumb for any one who has played the original game, and would surely have been obvious to both the developer and the QA team.
We're still continuing testing of PlayStation 4 Pro titles and plan to roll out coverage on each as soon as we can - though we've been limited by console availability. You can check out our YouTube let's plays for initial impressions on titles as we test them, but while only a selection of titles appear to be affected, we feel strongly here that this particular issue should be out in the open as soon as possible. PlayStation 4 Pro should in no way offer any kind of downgrade to any part of the experience in any title compared to the standard model, and while we realise that any hardware launch is going to be challenging for developers, it's really not a good idea to be bending the rules on Pro technical requirements when the system still has so much to prove. Performance is so closely linked to the quality of the gameplay experience and PS4 Pro really must match or exceed base PS4 frame-rates, as per the technical requirements.
While this is an issue we feel passionately about, we should remember that this is still early days and we've seen enough software that either matches performance or improves it while handing in visually superior results. However, more in-depth analysis is starting to show that even some 1080p Pro-optimised software may exhibits issues. Rise of the Tomb Raider gets everything right in terms of offering enhanced full HD modes and it looks beautiful at 4K. However, after a promising start in earlier levels, we're now finding that more challenging stages are causing issues - check out this shot. Base PlayStation 4 runs this at a locked 30fps with perfect frame-pacing. However, both 4K and enriched graphics modes are exhibiting really bad frame ordering, while the former is also running at a lower frame-rate in some cases compared to the original PS4 title. Unlocked frame-rate is faster, but not consistent. Those after a locked, consistent 1080p30 experience throughout Rise of the Tomb Raider are best off sticking to the standard PS4 right now.
With all of this in mind, it's worth stressing that plenty of titles on Pro are working as they should, offering enhancements and the same frame-rate or better, but we'd still like the option to run at base mode spec with Pro's CPU and GPU boosts fully unlocked. At the very least, being able to run with fewer (or indeed no) performance drops from the target frame-rate would be welcome. So, for The Last of Us, a plain and simple 1080p60 mode that guarantees performance in the way that no current version of the game can would be nice (maybe downscaling from 1440p instead of 1800p could work?) and would require little development work. Not every Pro patch upgrade needs to offer revelatory improvements - but there must be a fundamental guarantee that Pro will never hand in reduced performance compared to base hardware. The bottom line is that titles that do this serve to undermine the whole Pro concept.
We presented our findings to Sony yesterday and received this comment: "We are aware of the issue and currently investigating."