Digital Foundry vs. HDMI video

This one should be a no-brainer right? Everyone knows that HDMI is a digitally lossless form of transmitting audio and video, so it stands to reason that the cables - no matter how cheap or expensive - should either work or not work. There is no middle ground here.

But has anyone ever actually tested this, by directly capturing the traffic of the HDMI port using multiple cables of varying quality? Is there any actual need for this £655 HDMI cable?

Similarly, as the HDMI format is a purely digital form of transmitting the AV signals, the picture you get should be completely identical to the contents of the console's framebuffer, right? Well, that's something we'll be looking at later on in this feature - whether the promise of a digitally pristine video output is actually realised in the HDMI ports of the PlayStation 3 and Xbox 360.

First up: a bit of history. Just before Christmas, we retweeted this HDMI Truth Bomb "fact sheet" on the Digital Foundry Twitter feed, which led to a curious exchange with The Official PlayStation Magazine's Editor-in-Chief Tim Clark on how their tests seemed to bring up differences in performance of the cables used. A difference in colour depth, perhaps? The HDMI standard does support different colour formats (x.v deep colour, YPrPb 4:2:2, YPrPb 4:4:4, 24-bit RGB) but their test game, Killzone 2 - like all games we're aware of - is RGB only, so we were left a bit puzzled.

Human perception is a funny old thing. As the technology behind our gaming gets ever more complex, so does the make-up of the sound and the images being beamed to our senses. A native 720p console game running at 30FPS is pumping 28,648,000 pixels per second towards our eyes. Uncompressed 7.1 audio consists of 384,000 different samples of sound transmitted in the same period of time (yes, even the .1 channel gets full 48000Hz output just like the other seven channels - we checked with Sony).

It's all insanely cool, of course, but comparisons along the lines of using one screen with two different cables won't really work. Two identical screens, calibrated in precisely the same way, running side by side... Maybe that would be enlightening, but it's still somewhat unscientific. Human perception is obviously important, but it can't be measured or quantified.

Frame-rate analysis was one of our answers to the frailties of human perception - a bit of science to give us accurate, fool-proof data on the performance level of the games we play. So can the same thing be done to test the throughput of HDMI cables? Well, the principles of frame-rate testing - analysis of HDMI data - allows us to do exactly that. All we need to do is capture the exact same video output using a range of HDMI cables and use something like a basic MD5 hash checksum to make sure that the image is completely the same.

Our solution? Take a lossless 24-bit RGB screenshot of a 1080p game (Gran Turismo 5 is the obvious choice), then run it through the PlayStation 3's photo viewer. We used our stalwart "fat" 60GB NTSC launch model for reasons that will become clear later.

Initially we tried just pausing the game, in the hope that this would give us the same consistent still image. However, even though pause is in effect, the renderer is still running and producing microscopically different frames: invisible to the human eye, but not to our testing tools - so there is no chance of our mathematically precise checksums working out. XMB photo viewer it is.

But what HDMI cables should we test? Pre-Christmas we bought in a couple of sub-£1.50 cheapo leads from Amazon. The Digital Foundry coffers couldn't quite extend to spending £655 on an HDMI cable, let alone let alone three grand on the 5m version, but we did snaffle a Monster 1000HD 2m cable, which seems to cost anything up to £72 or even £100 (!!) depending on just how crap you are at using Google.

Acting as a bridge between the cheapo cable and the monstrously expensive one, we dropped in the official Microsoft 360 lead - £29.99 with audio dongle - and Sony's equivalent PS3 offering, which will set you back about £18 from numerous sources.

The cheapo Amazon cable certainly feels thinner and of a significantly lower quality than the others, while the Monster cable has some lovely plugs and a reassuring level of thickness and weight to it. The Microsoft product is, well, grey... but otherwise unremarkable. All of these are gold-plated, but Sony hasn't bothered with this for its official cable - and any engineer worth their salt will tell you that the advantages of gold-plating are dubious to begin with and pretty much completely pointless on a digital lead like this. There's a refreshing sense of functionality about the official Sony cable - yes, it's over-priced at £18, but at least it's 3m long, whereas 1.8m or 2m is more often the norm.

Onto the testing: the PlayStation 3 was set to output the same still image and it was duly captured at full 24-bit RGB precision at full 1080p via our TrueHD capture card. The process was repeated with each cable. Random frames from the captured streams were dumped, and hashed using freeware tool HashCalc. Just to be clear here, the XMB Photo Viewer is still outputting 1080p at 60 frames per second (with an uncompressed audio stream) - effectively the maximum throughput you will require for gaming using these cables.