Talking Project Morpheus with Anton Mikhailov

It's been nearly two years since John Carmack invited us into a room at E3 to show us Doom 3 running on prototype virtual reality hardware, but it was the insanely successful Oculus Rift Kickstarter that really suggested that the ancient VR concept might finally be ready to deliver on its promise. (Carmack clearly felt it was - he left id Software last year to join Oculus full-time.)

At the Game Developers Conference in San Francisco this week, Sony got in on the act, unveiling Project Morpheus, a PlayStation 4 head-mounted display that works with PlayStation Move and the PlayStation Camera to do many of the things Oculus has been celebrated for since those Kickstarted dev kits first started making their way out into the world. The results are already impressive, and Sony has obviously been working on it for a while. To learn more, we sat down with Sony R&D wizard Anton Mikhailov.

Eurogamer: You're not the only virtual reality unit out there and you won't be the last one. Can the market sustain many more?

Anton Mikhailov: Oh yeah, I mean there's not enough. Realistically, right - first of all we don't have a consumer version from any company yet that's taken off. In that sense there's not enough. But I think that nobody really worries about having too many TVs.

Eurogamer: Is what we're seeing at the moment running on PS4?

Anton Mikhailov: The Castle and The Deep, those are running on PS4. Eve Valkyrie and Thief are running on PC. They're special builds for Morpheus, but I think you can get a sense of what's possible with the PS4 with The Deep and The Castle, both of which are running at 60fps - and actually higher than 1080p in some cases.

Eurogamer: With Oculus, it's running from a platform that gets more powerful year after year, whereas you have a fixed platform. Do you see that as a problem?

Anton Mikhailov: We're in this for the long haul first of all. One thing people forget is what they've seen from PS4 so far have been the launch titles - if you look at the difference in quality between PS3 launch and PS3 end of lifecycle, the quality difference is massive. The advantage of fixed hardware is, while it doesn't revise every year, developers continuously find new tricks to employ because it is a fixed platform. There's a duality there. What you're seeing currently is a pretty compelling VR experience, and these are tech demos that haven't been optimised by top games studios. I think we're in a good place.

Eurogamer: High refresh rate is something that's important for VR - is it going to be a problem reaching that? You've got Thief here, and that struggles to reach 30fps on PS4 as it is.

Anton Mikhailov: Thief is... The build we have for Thief is running 60, but it's a PC build so it's not really a comparison. But The Deep and Castle are both running at 60, and there's other games on PS4 that are graphically intense that are running at 60. Really 30 or 60 on PS4 is more an aesthetic choice than a technological choice. Of course you can squeeze in more graphics at 30, there's no doubt, you have double the frame time. The reality is that when you're rendering VR, a lot of the details that traditionally get done in games like post-effects, motion blur - they're not needed. Things like a lot of the lens effects, a lot of the distortions people do, they're quite expensive to do in post. You don't need those in VR, because you're trying to make a more believable world. So a lot of what people do in current games are camera artefact effects - vignetting, film grain noise - so you save a lot of processing power there first of all.

"There's also another belief that people think it takes twice the power to render stereo - that's not true because your physics, your AI, everything runs once when you're rendering stereo."

Anton Mikhailov

There's also another belief that people think it takes twice the power to render stereo - that's not true because your physics, your AI, everything runs once when you're rendering stereo, and a lot of the graphics even only run once. For example, shadow computations can be re-used from eye to eye. So it's really not a 2x jump going from mono to stereo. I think it's quite feasible to run 60 in stereo for Morpheus on PS4.

Eurogamer: You've got very similar specs to Oculus. One big differentiation is the field of view. How did you come to that decision?

Anton Mikhailov: Is the difference diagonal or horizontal? That's the key there - diagonal is basically 1.4 times the horizontal. Ours is 90 degrees horizontal. If you do that calculation diagonal it's over 100, or somewhere - I think it's quite complicated doing the maths because the optics we're using are fairly non-standard, so I can't give you an exact answer. But it's certainly far above 90.

Because this is the wild west of VR, we don't have a standard way of measuring things. When you buy a 46-inch TV, you know they mean diagonal, not horizontal. If we'd like to compare specs, we need to get a very clear spec in line. And actually optics are even more complex than that - you know, for the head mount displays it's a little strange because the aspect ratio might not even be 16:9. What you really want is a vertical field of view and a horizontal field of view. Diagonal can be kind of misleading. It gets complicated, and the numbers range wildly - basically we can quote numbers between 90 and 120, depending on how you want to talk about it.

Another thing is glasses and eye relief. When you get closer to the optics in VR displays, you get a wider field of view. So if you're quoting a number that's at the lens, that could be quite a bit wider. The specs we quote is 90 degrees field of view for a glasses-wearing person at 15mm eye relief or further. So it's a very specific spec.

Eurogamer: How do you accommodate people with glasses?

Anton Mikhailov: The front optic block can move in and out, and there's space for glasses. But the optics are also designed so you don't mess up the optical things that glasses deal with. I'm not an optical guy so I can't explain it, I kind of know the math but I'm not at the level where I can sort of synthesise it for a mass audience. Basically there's a few things to do to accommodate spectacles or glasses, and we've talked to a lot of doctors in the field so that people with wide-range prescription glasses can use the device comfortably - they can focus at the right distance.

Eurogamer: One easy solution to get a lot of content on there from the off is porting PS3 stuff across, where you can easily achieve 1080p60 on the PS4. Is that something you're looking at?

Anton Mikhailov: I think the bottleneck there is not the frame-rate, it's the game design itself. You need to re-architect those games to make them valid experiences in VR. But you can take that content from those franchises, and that would be something to look into.

Eurogamer: You've got the Move controller working with it, and it works well - when the Move controller was being designed, was VR already in mind?

Anton Mikhailov: VR's been sort of a thing in our office forever. In US R&D we've had people that have worked with some of the earliest systems out there. We have a lot of people from schools like UNC which are well known for their tracking and VR simulation stuff.

When we were making the Move controller, it was obvious to us that if we're making a motion controller we should make it VR-ready. At the time we were making it, there wasn't really a clear way to make a consumer-level VR system. That's been coming together in the last few years due to technologies lining up and so forth.

In the back of our mind we thought it'd be awesome if some guy in a university used the Move with VR tracking systems, and they did. A lot of people used the Move at a University level to create VR apps, and we were thrilled with it. It was certainly specced - and you may argue over-specced - with the PS3 in the sense that it could do this VR thing. It's definitely been something in the back of our mind.

We do a lot of VR experience in our studio in general, not even using head-mount displays but using a technique known as fishtank VR where you look through a stereo display but it's sort of head-tracked. That sold us on Move interactions in a virtual world. That was through a window, right, and at that time we could only imagine what it would feel like in a virtual display, but it was obviously very exciting to us.

Eurogamer: You've got the audio-visual thing down, but a huge part of the immersion comes from the input. Move is brilliant, but it is quite dated in some ways too. Are you looking to refresh it to relaunch alongside the headset?

Anton Mikhailov: Move has a 15 million install base - there are a lot of them out there. It's very good for developers to have a platform to work from, the kind of experiences you get are dependent on the install base you've got. Sure there are indies who are willing to experiment with very niche devices, but that's something that's super-exciting for us in general, and that's why we've been engaging with indies lately. But a lot of the bigger studios don't want to jump onboard if they don't have an established device. We have to weight a lot of the advantages of creating something new versus supporting an established platform.

The reality is that Move is quite good at doing VR, as you've hopefully seen during the demos. There's no real technical reason for us to switch as of now. We filed a lot of patents when we did Move about some crazy ideas, some crazy knobs and dials that we'd put on Move, so we have a lot of ideas of how we'd change the device, but there are a lot of product discussions that are surrounding that.

Eurogamer: Oculus announced the pre-orders of DK2 this morning. How are you going about getting Morpheus prototypes in front of as many developers as you can? Are you going to be as open as that?

Anton Mikhailov: For dev kit details you'd have to contact dev support - I don't want to give out any false info. But I do know that we've been giving earlier dev kits to the ones we're showing today to internal and third-party studios, and also some indies have some dev kits. We've been very open internally, and obviously this is the first public showing we've had. I don't have the exact dates on when this kit will be available, but it's fairly soon.

Eurogamer: How far off is this headset we're seeing today to where you see the final product?

Anton Mikhailov: I think the honest answer to that is we're going to have to wait and see. We basically are building these things rapidly. We just got to a point with this dev kit where we thought, okay, we've got a good handle on audio, we've got a good handle on video, on input and on a lot of these other issues surrounding content, a lot of these experiences work. We don't have answers to all of these things right now, but we know of improvements in pretty much every single field. And so the real question is at what point do you say this is good for version one.

This is a long haul - there will be multiple ones of these in the future - and that's more of a product discussion. We don't have any info for that right now.

"[Motion blur is] really down to the LCD panel that we're using. I believe Oculus is using OLED. It's one of those things that is what it is in this dev kit. OLED is something that we're researching. Again, it's a question for a future dev kit."

Anton Mikhailov

Eurogamer: I've just come from Oculus, playing Valkyrie, and straight on to Morpheus. The two are very comparable experiences, but the one area Morpheus is behind a little is the motion blur. Is that something that you're aware of and are hoping to address?

Anton Mikhailov: That's really down to the LCD panel that we're using. I believe Oculus is using OLED. It's one of those things that is what it is in this dev kit. OLED is something that we're researching. Again, it's a question for a future dev kit.

Eurogamer: Are you experiencing issues with nausea at all?

Anton Mikhailov: Actually, this is the first event we've done in public - we've done internal events. So far, it's hard to get an accurate sampling, I haven't heard any instances of nausea from day one. We're not just accidentally having that. One of the key things of doing that is positional tracking. I believe a while ago I think Oculus announced they'd solved nausea issues in their dev kit, and one of the biggest jumps there was because the original dev kit did not have positional tracking, and the Crystal Cove one did. The minute movements in your head, if you don't replicate those, even though they're not very major, they cause discomfort and they cause motion sickness.

Of course there's latency, and it's kind of an obvious one - it's very easy to crank latency up and say, oh yeah, I feel bad. We've got a handle on latency, we've got a handle on position-tracking, and high frame-rate is important. Those are the simple ones. The really tricky ones are calibration - making sure your optics and your world are believable and lined up. Those are tricky, and it's not easy to even figure out if you've done it right, so you know there's a lot of interesting methods to figure that out. They're finding similar things that we're finding, and that Valve's finding. The more of these things you get right, the more you eliminate motion sickness.

Having said that, there are still experiences that are fundamentally more intense than others. If you go on a plane and you barrel roll a plane in real life, you're probably going to experience nausea. A plane pilot may not, because they're acclimatised and they've gone through training in pilot school. So what you're seeing is some of these people who are like the VR cadets - they've gone through the worst of what a VR demo has to offer, they're battle-hardened and they don't get sick at all.

There's going to be that crowd, and they're going to want the most intense rollercoaster ride. That might make people sick, even on good hardware. Even if you took the best VR hardware with the lowest latency, the lowest persistence, highest frame-rate, and you spin the person in a barrel roll like you would in a NASA centrifuge, they're not going to feel so good about it.

A lot of it is hardware, and a lot of it is content. We've got a good handle on the hardware. The content is up to the game developers. We have a lot of rules we follow internally to make sure that things don't go awry, but ultimately it's going to be up to the people, what kind of experiences they want, and the developers, what they want to create.

Eurogamer: I thought some of the most exciting parts of the reveal were the non-gaming applications. For that to really happen on a large-scale, are you going to have to go beyond PlayStation?

Anton Mikhailov: What we were discussing there was the vision of VR as a whole. At this time we're focused on PlayStation. Those experiences can be done on PlayStation, but like you said - I mean, PlayStation right now has Netflix, and a lot of people like it because they can watch TV on their PlayStation. In some sense it's not so alien to have non-game apps on your PlayStation. For the broader VR, there's going to be apps out there, they might be part of the PlayStation ecosystem they might now.

That's orthogonal to the philosophy aspect of it. The talk was more focused on how VR is a medium that's exciting not just for games, and that's going to help its adoption. I think we're going to push in some of those areas - not necessarily all of them - and even non-gaming apps can go on PlayStation. And who knows, in the future there might be other platforms.