PlayStation 3: retail disc vs. digital delivery

We're moving ever closer to the next-gen era where console games will be available to buy via digital delivery, day and date with the retail disc versions, but in the meantime it's Sony that is leading the charge in providing timely PSN releases of many of the latest PlayStation 3 titles. They're not cheap and you can't resell them, but download titles do offer up the enticing possibility of running your games entirely from hard drive, divorced completely from the much slower Blu-ray optical drive. So the question is, do PSN downloads provide a superior experience to traditional disc-based gameplay?

It's a topic that often comes up during our Face-Off coverage. Xbox 360 lets gamers download their discs to hard drive, at the very least providing some kind of loading time advantage - but to be honest, it doesn't offer much else beyond silencing the noisy DVD drive. Meanwhile, PS3 owners are very much at the whims of the developer as to whether or not data installs to the hard drive, with only a tiny fraction of titles supporting optional installs. Utilising digital downloads from PSN returns the choice to the user but more than that, PS3 gamers can use any hard drives they want, even allowing for the introduction of solid state media - SSDs - for the very fastest possible performance.

But what does that that extra performance actually translate into during gameplay? Beyond loading times, do we see any improvements to frame-rates, geometry or texture pop-in by embracing digital delivery? We decided to find out, utilising our available library of PlayStation Plus "instant game collection" titles, compared against the retail disc versions of the same game. Our focus titles are Mass Effect 3, Sleeping Dogs, Mortal Kombat and MotorStorm Apocalypse - a decent range of games that should allow us to test both loading times and in-game asset streaming.

The friction-filled experience

Quite apart from the fact that buying a retail disc gives you reselling rights and a much easier way to archive your games, there's a significant amount of friction involved in actually getting any PSN game downloaded and running on your system. Part of that is down to PSN not being as fast as it should be: we carried out our testing on an "up to 20mbps" ADSL connection but our throughput topped out at 12mbps max - pretty much 1.5MB of data throughput per second. On what we assume was a congested period, the 14GB Mass Effect 3 download slowed down considerably, taking a mammoth six hours, 17 minutes to complete - a disappointing 5mbps. On the same connection, Steam locks to 16mbps and with a level of consistency that PSN can't match.

Next up there's a fundamental difference between Xbox Live, Steam and PSN in terms of what happens when data is decompressed after it has downloaded - with Sony's chosen technique being a bit of a pain in the arse. Steam appears to decompress data as it arrives so your game is good to go pretty much as soon as the download is complete, while Xbox Live doesn't do any decompression at all - games are delivered in a Microsoft equivalent to the ISO format - a simple, big archive file. PSN offers the poorest solution of them all, decompressing the data only when the download is complete.

Depending on the size of the download and the amount of files within, installs can take a very long time indeed. It also means that you need extra space available on your hard drive - which needs to host the original download and the uncompressed version (again, these are not limitations on Xbox Live or Steam) until decompression is complete. So let's see how all that downloading and installing breaks down in our chosen titles. In terms of the install times, we actually carried out the process twice over: firstly, by using the standard 60GB HDD supplied with the EU launch PS3, and secondly we repeated the procedure with an SSD installed.

Obviously, the bigger the game, the more data there is to decompress and the higher the likelihood that there'll be more files to extract, but suffice to say that we're glad that Sony has completely rethought its strategy for digital downloads for PS4. Clearly, the current situation is rather grim. On a standard PS3, you're looking at anything between 19 to 38 minutes of waiting before you're able to access the games that already took so long to download, on a service that clearly has real issues utilising the full bandwidth of your connection.

What's curious here is that the SSD results are fundamentally much poorer than they would be were we to carry out the same decompression tests even on a modest PC. We have a couple of theories about why this might be the case. Firstly, all previous testing we've carried out has suggested that PS3 bandwidth on sequential read/writes is limited to around 12-15MB/s, no matter what kind of HDD or indeed SSD you utilise. This nullifies much of the advantage you get by moving across to solid state storage where even the poorest SSDs operate at least ten times as quickly in PC hardware.

The only advantage SSD has is the virtual elimination of seek times - the process of moving from one file to the next. On a traditional hard drive, the head moves across the surface of the disk - a process that takes time. The same action has no latency whatsoever on an SSD, speeding up the process considerably, depending on how many files there are - the more files being accessed, the more the SSD commands an advantage. Quite why Sleeping Dogs was actually slower on SSD is a bit of a mystery that we can't readily explain - all our other results point to the solid state drive in offering a noticeable, though not quite spectacular, advantage.

Running from HDD - does it improve gaming?

Developers have the option to offer users the ability to dump a section of game data onto HDD in order to improve performance - Ridge Racer 7 was the first game to support it, and Gran Turismo 5 is one of the games that benefits the most from it. Now, the question is, why don't more game-makers offer that functionality? It's an interesting point but our investigations on our selection of test games suggest that while there are loading time advantages to optional installs, improvements from a gameplay perspective are extremely thin on the ground.

That's not to say that they're completely non-existent though. Take Mass Effect 3 for example. The disc version doesn't actually install anything to the hard drive (though we suspect that the HDD cache gets a thorough workout). Now, generally speaking, texture pop-in isn't really an issue but in areas where a new location is loaded, we do see textures resolve more quickly on HDD and faster still on SSD. There's a similar effect on Sleeping Dogs, but nothing really noticeable across the run of play, while on MotorStorm: Apocalypse, some elements appear to arrive on-screen a split-second faster on the PSN versions compared to the disc game. Of course, Mortal Kombat loads straight to RAM and has no streaming that we're aware of, so there's no appreciable in-game difference.

Launch comparison tool

We're fairly confident that the vast majority of games play just as well from disc as they do from a PSN download, which is perhaps not surprising. Developers build their titles around the bandwidth offered by the Blu-ray drive, and for those that need the extra speed, they tend to go for mandatory installs to the hard drive.

So effectively for the most demanding games that use an existing HDD install we are just comparing hard drive to hard drive - full install vs. partial install, if you will. It's not really that surprising that there are thin pickings here, and that the majority of the improvement comes down to loading times, particularly on games where there is no mandatory install.

Here's the lie of the land in terms of loading times across our test cases. There's a noticeable boost going from disc to HDD to SSD, but it's hardly game-changing stuff - certainly not enough for us to want to give up our rights to resell our games, or to more easily access them. That said, an OS-level HDD install option for all games would still be much appreciated.

So are there any instances where a hard drive or SSD upgrade can make a genuine difference you actually feel? Well, this is well-trodden ground and something that Digital Foundry has covered in the past. Our conclusion back then was that while faster drives do present some advantages, just like our test cases above, the vast majority of the improvement was in loading times alone.

However, there are a select few titles that truly hammer the hard drive and where moving to a faster drive or even an SSD make a genuine difference. GT5's prolonged loading and install times are improved considerably, for example, while id software's Rage - with all its texture popping issues - is significantly transformed by upgrading to an SSD, as this video demonstrates. Indeed, moving up to a hybrid SSD/HDD also makes a big difference but we're not quite sure if that's actually down to the SSD component or whether it's just taking advantage of the faster 7200rpm drive.

The much maligned Skyrim bug (now mostly patched) - where the game appeared to use the hard drive as an additional level of cache - also saw very noticeable improvements when the HDD was jettisoned in favour of an SSD. The same issue manifested in both Fallout 3 and New Vegas and was never patched, so if you're still playing those games and performance is compromised, an SSD could make sense. Over and above that though, the case for utilising an SSD in your PS3 is pretty shaky.

PS3 Skyrim "Rimlag" - Hard Drive vs. SSD vs. Hybrid - Patched/Unpatched

What happens with next-gen?

Everything changes with the arrival of the next generation Xbox and the PlayStation 4. Comparisons between disc and hard drive will become mostly irrelevant, as it appears that both Sony and Microsoft have decided that every game should install in its entirety onto the hard drive. To ease the frustrations caused by a generation of hanging about waiting for installs to complete, the strategy seems to be that games boot from disc and "stealth install" as you play them.

Developers have already experimented with this on the PlayStation 3 - Dragon's Dogma does it, as do a clutch of older Unreal Engine 3 engines (X-Men Origins: Wolverine amongst them). Hardware decompression is built into the silicon of both next-gen consoles, meaning that - fingers crossed - the notion of copying over thousands of smaller files to your hard drive (Gran Turismo 5 copies across over 8000!) in an uncompressed state will become a thing of the past.

The interesting new addition here is that the stealth install strategy appears to have been expanded to digital delivery too, where a section of the game downloads into RAM and you play that while the download/install continues in the background. We can't help but wonder whether standard ADSL broadband will be able to keep up here.

It's clear to see where the platform holders are heading here - it's all about removing the optical drive as a performance bottleneck, making use of the hard drive more extensively without leaving gamers hanging about waiting for time-consuming installs to complete. We'll be interested to see whether games can be divided up into chunks that are fast enough to download as seamlessly as the concept demands, but in theory this all sounds like the best possible way to remove the friction that makes the console digital download experience as frustrating as it is now. Let's just hope that both Microsoft and Sony allow us to upgrade the hard drives as we see fit, without imposing proprietary solutions.