The lines between online and offline shopping have already blurred. With companies like Amazon and Instacart innovating online commerce, shopping patterns have changed. Only a few years back, getting groceries and apparel delivered to our doorstep with the touch of a button was a distant dream -- now, it seems to be the norm.

Shopping online is convenient, fast, and stress-free. But it’s not all bad news for brick and mortar stores; they still have something that online commerce will not be able to provide -- a physical, in-store experience. The brick and mortar stores that best integrate the advantages of online shopping into their physical experience will stand the greatest chance of improving quality of engagement with their customers.

Innovative sensor technologies are already making their way into retail locations; several companies have been experimenting with ideas in geo-fencing, guided shopping, and precise, in-store, location-based deals through the placement of beacons.

At Braintree, we hacked together a quick prototype application for a touch-free kiosk where customers can browse a catalog and check out a product using only hand gestures using Leap motion controller. With advances in virtual/augmented experiences, we believe that very soon, retail store shopping experiences will change and that it should be possible to virtually try on anything from anywhere. In such experiences, it might be easier to interact using gestural interfaces than traditional keyboards. For instance, a swipe might change the color or type.

For this example, we used simplecart.js and bootstrap to create a demo store. One reason for choosing simplecart for this example is its readily-available PayPal integration.

The application responds to the following gestures:

• Swipe right - Selects the next product
• Swipe left - Selects the previous product
• Tap - Adds the current product to the cart
• Pinch - Checks out using PayPal

The code can be found here

Pinch gesture

Leap.loop(controllerOptions, function(frame) {

  if(frame.hands.length > 0){
    if(frame.hands[0].pinchStrength > 0.999){
      ... //trigger checkout
    }
  }
}

Swipe gesture

if (frame.gestures.length > 0) {  
    var gesture = frame.gestures[0];
    switch (gesture.type) {
      ...
      case "swipe":
          if(swipeDirection(gesture.direction) == 1){
            //do something for right swipe
            ...
          }
          else {
            //do something for left swipe
            ...
          }
       ...
    }
}

Tap gesture

if (frame.gestures.length > 0) {  
    var gesture = frame.gestures[0];
    switch (gesture.type) {
      ...
      case "keyTap":
       ...
    }
}

This deceptively simple app does lead to some interesting use cases around in-store shopping. For instance, a store can have a retail kiosk that allows browsing for variants of a product seen on the shelf. Theoretically, leap sensors can also detect which products are being interacted with and when combined with point-of-sale data, such information is valuable for analytics. When this technology is paired with image recognition, touch free payments are also possible, where the customer can trigger a payment action through gestures. These are early stages, but this is fascinating technology.

Hack on! Let us know if we can help you build cool commerce use cases using Leap.