Sticker Shooter: A Simple Leap Motion VR Test

I bought a Leap Motion several months ago, and in the haste and excitement of ordering it, I bought the version without the developer VR mount. Last week, though, I finally got a chance to sit down and play with it and created a basic “Sticker Shooter” app that uses Leap Motion gesture controls to shoot objects from the VR camera that stick when they collide with things in the environment.

Leap-Exploded-hero

The Leap Motion sensor components, (c) 2015 Leap Motion

So what exactly is a Leap Motion controller? Put simply, it’s a device that allows you to interact with a screen (or VR environment, in our case) using your hands and fingers, which are tracked with a combination of camera and infrared sensors and passed through to the given application as an input. One of my favorite demos that uses Leap Motion is ElementL, a game that lets you throw fireballs and water bubbles to ward off skeleton demons.

Elementlimagehands

ElementL: Ghost Story gameplay using a Leap Motion controller for hand inputs, LeapMotion.com

In this post, I’ll be sharing the process & code I wrote to get started with integrating Leap Motion into my own project using Unity.

Getting started with integrating the Leap Motion controller into a Unity application ended up being fairly straightforward. The general architecture behind the update and input control is something like this:

  1. Application initializes during setup and a Controller object is created
  2. The Controller generates a list of Leap Motion devices that are detected
  3. A listener is attached to the Controller to track events
  4. The Controller is set up to accept specific gestures from the gesture list (if you are using the predefined gestures from the Leap software)
  5. Within the game update loop, the Controller queries for frames and checks if specific gestures have been detected within the given frame
  6. If a specific gesture is found, set to perform action given which gesture was recorded

In my application, I created a general game play script and started off adding in two global variables for the script:

 Controller leapController;
 Listener leapListener;

In the main startup function for the application, I initialized the Controller and added the listener for the device (bonus debug checks included!). I then set up the gesture detection by enabling two separate types of gestures, a circle gesture and a tap gesture. These gestures are denoted by Gesture.GestureType.<TYPE>:

 void Start () {
 leapController = new Controller();
 leapListener = new Listener();

 // Log that the Leap Motion device is attached and recognized
 foreach(Device item in leapController.Devices)
 {
 Debug.Log("Leap Motion Device ID:" + item.ToString());
 }
 // Add a listener so that the controller is waiting for inputs
 leapController.AddListener(leapListener);
 // Check that the leap is fully connected and ready for input
 Debug.Log("Leap Motion connected: " + leapController.IsConnected);

 // Set up the gesture detection
 leapController.EnableGesture(Gesture.GestureType.TYPE_CIRCLE);
 leapController.EnableGesture(Gesture.GestureType.TYPESCREENTAP);
 Debug.Log("Circle Gesture Enabled: " + leapController.IsGestureEnabled(Gesture.GestureType.TYPE_CIRCLE));
 Debug.Log("Screen Tap Gesture Enabled: " + leapController.IsGestureEnabled(Gesture.GestureType.TYPESCREENTAP));
 
 }

Within the Update() loop for my application, I created a “Check for Gestures” function that was called on each frame firing. Since Unity calls Update() once per frame, and the Leap Motion controller also grabs gestures on a frame-by-frame basis, I call CheckForGestures() in Update() so that that gestures aren’t lost.

Within my CheckForGestures() function, I grab two frames from the controller object: the current one, and the previous one. Some of the Leap Motion gestures are tracked over multiple frames, so this would be used in checking for those. That said, the ones that I was tracking are detected in a single frame, so I didn’t end up using the previous frame yet. For each frame object, a list of detected gestures is provided, so I went ahead and grabbed the list and iterated through them to see if my desired gestures were detected. If they were, I fired a sticker:

void CheckForGestures()
 {
 if(leapController.IsConnected)
 {
 // Check the list of gestures for a circle
 Frame current = leapController.Frame();
 Frame previous = leapController.Frame();

 GestureList gesturesInFrame = current.Gestures();
 foreach(Gesture gesture in gesturesInFrame )
 {
 Debug.Log("Captured a gesture: " + gesture.Type.ToString());
 if (gesture.Type == Gesture.GestureType.TYPESCREENTAP || gesture.Type == Gesture.GestureType.TYPECIRCLE) ;
 {
 FireSticker();
   }
  }
 }
}

And that was it! With just a few lines of code, capturing and using gestures with Leap Motion adds a new level of interactivity into a basic point & click app!

Related Posts

Leave a Reply

Your email address will not be published.