HoloLens is a surreal experience - and full of possibilities
Can't touch this.
In 1895, the Lumière brothers made a short film about a train pulling into a station called L'Arrivée d'un train en gare de La Ciotat. Legend has it that people ran to the back of the theatre screaming as they thought the steam engine would smash into them - a scene popularised in Martin Scorsese's film Hugo. "Fools!" I'd think as they overreacted to something that wasn't even there. Then, earlier today, I did basically the same thing when I walked through a hologram of NASA's Mars rover.
I clipped through it of course. Of course! Right? But at that moment, even though I knew I was in an open space, I tensed up preparing to stumble over everyone's favourite space-exploring robot.
This is the future, ladies and gentlemen. Holograms are real and they do more than resurrect dead rappers for music festivals.
Yes, they do require wearing a goofy headset, so it will be a long time, if ever, before they'll be integrated into public scenarios - so don't expect Back to the Future 2's Jaws 19 ad to chomp its ghostly bite over your persons while milling about - but that doesn't mean they don't have practical uses. In fact, Microsoft has cooked up some pretty impressive benefits to this somewhat surreal technology.
But first things first, here's how Microsoft's newly revealed augmented reality headset, the HoloLens, is controlled: once donned, there's a cursor in the centre of your vision that moves with your head. Voice commands are self-explanatory, while "clicking" your cursor is handled by what Microsoft calls an "air tap," which is basically moving your index finger up and down, not unlike a certain creepy boy with a fondness for something called "red rum." You have to hold your hand in front of your face for it to work, as the camera on the HoloLens needs to see your digit. It's a little awkward, but not too unwieldy.
So how is it used? Let me count the ways...
The first demo I'm shown tasks me with calling up someone on Skype to help me install a light switch. An actual physical light switch. Naturally, I fit the HoloLens to my noggin and am told to call someone named Terry. The Skype app through HoloLens doesn't occupy my entire field of view, but is rather a smallish window making it appear like there's a 15-inch laptop screen a foot in front of me at all times. I tilt my head until the cursor highlights Terry from my contacts, perform an air tap, and the next thing I know he's on video in the window in front of me. Now here's where it gets interesting....
You see, Terry can see everything I see. Better yet, he can draw images into my reality. He tells me to look at the tools set before me, then draws a green arrow to the one he wants me to pick up and circles it. It's like a ghost is haunting me, only instead of trying to scare me he provides aid in menial tasks. Oh, and I can "pin" his window to a fixed location in the room so he's not always occupying my field of view.
Sure, it's a pretty mundane use of the tech, but I imagine it has more practical applications in the real world. What if I wasn't installing a light switch, but rather diffusing a bomb? That's a real thing people do. What if it's a customer service person helping someone with a computer error? As a simple tool for teaching, HoloLens is pretty impressive.
It's also a great tool for science. The most extreme example of this is a project Microsoft has cooked up with NASA wherein the US space exploration agency has set up a virtual reality representation of Mars based on a composite of images taken from its heroic rover Curiosity. Like the Skype app, HoloLens' image doesn't occupy my entire field of view, but it takes up the lion's share of it with my peripheral vision grounding me to reality.
But forget your peripheral vision for a moment, as your brain is bound to do anyway when faced with such a sight; this 3D representation of Mars is shockingly convincing. As I mentioned before, I slowed down upon apprehensively attempting to walk through the holographic rover in front of me. The strawberry blonde hue in the air is far more lifelike that the grainy images seen elsewhere, while I was free to walk around the three augmented reality sites manifested for this demo.
Sure, looking at this Mars simulation's surface is impressive, but it's not just for show. There are things I can do here as well. Like a GTA map, I can set waypoints with my cursor, which are useful for collaborating with other scientists when planning where to dig. At one point I try to break the simulation by placing my waypoint - a gold flag about a foot long - under a rock. Yes, I have to bend down on my knees and peer under the lip of a flat stone to plant my cursor here. Amazingly, it works! The flag goes through the rock, but it doesn't look like an error or a glitch. Instead, a gold circle surrounds every instance of the waypoint making contact with the scenery.
This is all well and good, but it gets better when I encounter a gold alien figure vaguely resembling a Giygas cosplayer. Turns out this statuesque bloke is actually the avatar of a fellow scientist. He has a waypoint of his own, in the form of a dotted line leading from his eyes to a spot in the ground. Together, we discuss where the best spot would be for Curiosity to collect a sample of debris.
There's another peculiar instance of augmented reality coming in handy here. Looking at a screen while wearing HoloLens breaks the image, so you can make out a computer monitor clear as day. While I look at a 2D black-and-white photo of Mars' surface, a Microsoft rep encourages me to move the mouse to the edge of the screen. Confused, I do and nothing happens. "Move it further," the rep tells me. And lo and behold, the mouse' cursor exists offscreen in the Mars simulation. It's as if I'm living in someone else's point-and-click adventure game. Isn't this how The Dig started out?
One potential limit I see is that I can only walk about eight feet in any direction before I'd bump into a wall. We're not in a huge room, after all. "What if I'd like to explore further out?" I ask. A Microsoft rep assures me that there will be a way to shift the scenery over without physically walking there. He's mum on the details, but I reckon it will probably let you move around via analogue stick like a first-person shooter or let you scroll around a 2D map and refresh the image a la Google Maps. Either way, there will be a method to extend the scale of the simulation.
The most game-like application for HoloLens might have been my favourite. Dubbed "Holobuilder," it's an augmented reality spin on Minecraft. Placed in a cozy furnished room vaguely resembling a therapist's office, the HoloLens prototype is strapped to my head and suddenly the room is decked out in Minecraft structures. The coffee table, counters and shelves are adorned with castles, cottages, rolling green hills and the occasional little green creeper. So far, so Lego. The difference is I can manipulate not only the additional virtual structure, but the surfaces of real world objects.
For example, a bench's surface contains a tower on one side, a gaggle of orcs on the other, and a patch of grass in the middle. The space under the bench is a subterranean cavern that's all dreary grey walls. When I say "shovel" my cursor turns into the tool in question and by air-tapping on the patch of grass I'm able to protect the tower from the invading orcs. More impressively, I can now see through the hole in the bench. There are even shafts of light peering through it when I investigate the landscape below. It looks as if I should be able to put my hand through the bench's hole. Obviously I can't, but this doesn't stop me from trying. What? How? Magic!
This same technique is employed when I encounter a shelf full of explosive barrels. I switch my icon from a shovel to a torch and ignite these barrels only for them to blow a hole in what otherwise looks like a real, ordinary wall. This opens up a passage into a cave full of lava fountains and bats who flutter out of this newly discovered chasm. I quite literally broke the fourth wall. Heck, I could even place torches into this unreal space to illuminate it further.
A Microsoft rep notes that HoloLens analyses the surrounding space in real time. In this way it's not dissimilar from how Kinect scans the spatial layout of your apartment. Just imagine playing an "escape the room" game tailored to your own apartment with keys and puzzles randomly hidden throughout your flat.
The final demo I see is, regrettably, a hands-off presentation, but it's still impressive in its potential. Already shown off during Microsoft's Windows 10 press conference, Holo Studio is the holographic equivalent of MS Paint. A user puts on HoloLens and selects different icons to create shapes, rotate them, alter their size, change their colour, and draw designs on them. Plus you can physically walk around your creations and view them from any angle.
The demo we see has a Microsoft employee put together a cute koala toy in a matter of minutes. According to the sculptor in question, he'd had zero training in 3D animation or artistry of any kind. Yet this man supposedly put together a nifty model of an X-Wing in approximately 90 minutes.
Simply creating holographic figures is one thing, but you can then send them to a 3D printing shop to manifest your creations concretely. Holo Studio's creative director Cam "We're Not Using Last Names Today" Something said he assembled an astronaut koala toy as part of an in-joke with his young niece in Australia. He then printed it at a local shop and sent it overseas to her as a birthday present.
Of course, it seems like it would be much cheaper to print it in Australia to cut down on shipping costs. I ask if you'll be able to send your hologram across the globe and Cam confirms that this will be an option. (He just didn't do it that way because he had some sort of shipping discount set up. Fair enough.)
It's worth noting that there are some bugs in the holographic ointment that still need to be sorted. The prototype units we tried on required Microsoft staffers to measure the distance between our pupils before fitting; we're told the final unit will do that automatically. Furthermore, while the units can fit over glasses, this is far from an ideal solution. I didn't even bother as I didn't want to risk scratching my lenses, while I overheard another person complain that it pressed her glasses into her nose too much and she found it more enjoyable eschewing her spectacles altogether. My vision isn't terrible - I'm somewhat near-sighted with astigmatism - and I found it worked mostly very well with the exception of trying to read text during the Skype demo.
But that's not bad! Comparatively, the first time I tried Oculus Rift it left me quite ill (as it has continued to do in the times I've tried it since). HoloLens, however, didn't make me even the slightest bit dizzy. I could walk, crouch, look around, stroll through illusions that weren't there and touch seemingly invisible walls. Based on my dozen or so minutes with HoloLens, I'd say it's strange, surreal and full of possibilities. And unlike in Robert Zemeckis' prediction of 2015, the holograms here don't look fake. It may be over a century since L'Arrivée d'un train en gare de La Ciotat, but we're still finding new ways to make people flinch.