Performance Analysis: Call of Duty: Black Ops 3
Does Treyarch's most ambitious campaign yet compromise the core COD experience?
You've got to hand it to Treyarch - the team's Call of Duty titles always push boundaries in terms of scale, scope and spectacle. Blacks Ops 3 delivers yet another bombastic Call of Duty campaign, this time backed by four-player online co-op, along with a two-player local split-screen mode. The team's technological ambitions are also remarkable in terms of the game's visuals: physically-based rendering, simulated global illumination and a battery of impressive post effects work makes this new campaign the most visually arresting yet.
But has the developer perhaps pushed its technology too far? The campaign co-op demo revealed at this year's E3 had us slightly concerned: frame-rates throughout the demo often fell short of the desired 60fps target, even dipping below 30fps. On top of the optimisation effort, it seems that the team's solution to the performance challenge was to implement yet another ambitious piece of technology: dynamic resolution scaling. This involves adjusting pixel-count on the fly, depending on the engine load - it's a way of freeing up valuable GPU time without permanently compromising on image quality, and produced some remarkably smooth results in the recently released Halo 5.
So how does the dynamic scaler work in Black Ops 3? Both Xbox One and PS4 versions render in native 1080p during cut-scenes, and for the most part full HD quality is retained throughout the duration of these sequences. Anti-aliasing is handled via the use of filmic SMAA, successfully providing good coverage across geometry edges, though some texture blurring is apparent. PS4 doesn't appear to drop below 1080p at all in these scenes, whereas the Xbox One version seems to transition down to 1728x1080 just before gameplay kicks off - and that's where the scaler really gets to work.
Once the player gains control we see shifts in native resolution across both platforms, according to engine load. PS4 ranges between 1360x1080 to 1920x1080, although much of the time the engine manages to hit the desired native 1080p resolution for extended periods. During the opening firefight in the Provocation mission we see PS4 kick off at 1360x1080p before ramping back up to full 1080p a few moments later - the switch is often barely visible due to softening effect of the AA solution, although some blur across distant details is apparent.
Xbox One is a different story, targeting a baseline 1600x900 for gameplay, but after trawling through our captures it appears that the engine rarely - if ever - achieves this. Instead we're looking at a sustained 1280x900 resolution, even in less stressful gameplay scenes, with horizontal metrics dropping down to 1200x900 in more challenging scenarios - and the results are not impressive. Due to the heavy upscale, the Xbox One release possesses a consistent blurry appearance when the action really kicks in: fine details are frequently smudged over, while geometry edges appear rough and fuzzy. It's a substantial downgrade over last year's Advanced Warfare, where Sledgehammer stayed locked at a 1360x1080 resolution during the more demanding scenes.
We could accept the compromised image quality if the end result was something approaching the locked 60fps we expect from a Call of Duty title, but unfortunately, the dynamic scaler doesn't quite provide the frame-rate uplift the campaign really needs. Performance is the real challenge here: 60fps is the target, but the tight focus we saw here in Sledgehammer's Advanced Warfare isn't quite so pronounced now. Previous Treyarch campaigns have seen the lead platform achieve what we sometimes describe as a perceptual 60fps - frames are dropped, but the fluidity and the controller response aren't unduly compromised. The issue with the Black Ops 3 campaign is that 60fps is the exception, rather than the rule.
Gameplay set in narrow corridors, along with low key gameplay moments away from frenetic combat can see the engine attain the familiar 60fps expected from a Call of Duty title. However, as we move into more open areas packed with greater amounts of geometry detail and more challenging effects work, frame-rates start to suffer on both consoles and the 60fps fluidity is gone. Performance ranges from 40-55fps and seem highly variable in motion, leading to inconsistent controller response and a stutter to motion we don't usually associate with the series.
Screen-tear is also present too - but it's limited to just the top of the screen. Treyarch aims for the visual integrity of v-sync, but allows some flexibility in the render-time budget in order to reduce latency. It's a technique that's becoming increasingly popular in modern titles (Rise of the Tomb Raider does something similar) and in this case, the flickering at the top of the screen can distract. Top-end frame-rates are usually higher on PlayStation 4, but under load, performance can be decidedly wobbly on both platforms.
On the face of it, the raw metrics we drew from our captures don't look so bad - PS4 hands in an average frame-rate of 51fps, Xbox One is at 49fps. From our sample, around 15 per cent of the video output consists of dropped frames on the Sony console, rising to around 18 per cent on Xbox One. Arguably that's already too high for a Call of Duty game, but it's drawn from the complete sample and not really representative of in the moment gameplay - there, it's the sudden, often jarring changes during gameplay that can compromise the experience.
And it's Xbox One where we see the most extreme scenarios where Treyarch's precarious balancing act doesn't really work out. A frantic shoot-out defending a control room is a low point: while PS4 sticks close to 60fps at various moments throughout the battle, Microsoft's system frequently struggles to rise above the 35-45fps No Man's Land, eventually succumbing to a 28fps drop. The situation here is grim: judder and fluctuations in controller response compromise the gameplay, while the dynamic resolution plateaus to its lowest point. It doesn't look great, but more importantly, the feel is wrong.
Adding to the sense of inconsistency is the use of cut-scenes capped at 30fps, which instantly segue into gameplay with the frame-rate unlocked. On the plus side, image quality increases here - both systems render these sections at 1080p. However, it's not just the cinematics that run at the lower frame-rate: every non-playable sequence does, even to the point where interacting with levers and buttons, breaching into rooms and getting in and out of vehicles sees this sudden, jarring shift in the update. It's another example of how the visual language of Call of Duty has changed - and not really for the better.
And that's a shame. In many ways, Treyarch's ambitions for the Call of Duty campaign are a revelation for the series - it's still a limited, linear experience built on spectacle, but the ability to share the experience with others and to access any area you want, whenever you want (servers permitting) is a game-changer. But perhaps the team has pushed too hard here. In many senses, frame-rate is gameplay, and that's especially true of Call of Duty, where the interface between player and game relies so heavily on crisp response and low latency controls. In the Black Ops 3 campaign, fluidity is compromised and in the case of Xbox One in particular, image quality suffers badly when performance is at its worst. PlayStation 4 offers an improved overall experience, but there's the sense that the balancing points still aren't quite right.
In closing, we feel it's worth stressing that this article relates exclusively to our experiences with the campaign section of the game - traditionally the technological showcase of any given COD title. We've just begun to dig into the multiplayer component and what's immediately and obviously apparent is that performance there is much, much closer to the locked 60fps we expect from the series - and very much in line with the positive experiences we had with the beta code a couple of months back. For many, this is the true core of the game, and it's good to know that the gameplay here doesn't have anything like the same level of compromise as the campaign. We'll have performance analysis of this area of the game in our upcoming Face-Off, plus we'll be taking a closer look at the contentious PC version too.