As impressive as virtual reality can be, you’re still left holding uncomfortable fashioned controllers in your hands – a reminder for you that you’re very much in the actual world. But what if as an alternative to hitting A to move a rock, you could simply use your hands? That’s precisely what Leap Motion, known for its hand-gesture control sensors, has been working on for the past few years. Most lately, it’s been at work on something called the Interaction Engine, which tends to take things a step further: to make picking up objects in the digital world sensation as normal as it does in the real one.
I got a chance to attempt the latest demo for myself last week. Foremost, I wore what was fundamentally an Oculus Rift DK2 with one of Leap Motion’s 3D motion sensors attached to the front. While trying the headset, I could perceive my real hands in front of me, at which point I could shove virtual boxes out of my way merely by waving them around. There was barely any latency — Leap Motion says the camera streams about 120 frames per second — which I found impressive. Still, this was a demo that I’ve seen before, and it wasn’t too difficult to figure out.
What came next is one of some limited new actions aided by the Interaction Engine. I was asked to hand-pick a single ball from a grid of virtual spheres and then drop it in a cage. I pretended that I was picking up a real physical object, transporting it to its destination and then letting it go. The cage ignited up and made a sound when I was finally successful. Caleb Kruse, who assists user testing at Leap Motion trained me to put my thumb and forefinger mutually with both hands. Then he instructed me to do a “pinch out” gesticulation while stirring my hands apart at the equal time. Immediately, I realized that I could create blocks. As I kept on doing it, I loaded one box after another, forming a tower. Then I collided it down with a wave of my hand. Moreover, I could turn a block on its corner with one hand and then rotate it with the other.
This demo requires accurate hand-tracking as well as a robust graphic engine. Obviously, this is still firmly a demonstration, planned mostly to inspire developers. I was overwhelmed by how simple it was to get my hands in and start generating virtual objects. Buckwald said that controllers still have their place in virtual reality, particularly for driving or combat-style games. But for more experimental uses, hands are quite better, since they allow you to gesticulate, point and pick up items without a second thought.
The next stage for Leap Motion is to see if it can grow this technology in consumer headsets. Buckwald thinks that upcoming VR headsets will be much more portable: Instead of being tied to a PC or demanding a phone like the Gear VR, all the processing would be completed in the headset itself. This would also make it cooler to merely integrate Leap Motion’s sensors into the hardware. Moreover, it makes much more sense to make use of motion controls when you’re mobile.