CHI 2021

SoloFinger: Robust Microgestures while Grasping Everyday Objects

Adwait Sharma, Michael A. Hedderich, Divyanshu Bhardwaj, Bruno Fruchard, Jess McIntosh, Aditya Shekhar Nittala, Dietrich Klakow, Daniel Ashbrook, Jürgen Steimle

In short

Microgestures can let people interact with devices while their hands are occupied, but ordinary finger movements can accidentally trigger commands. SoloFinger addresses this problem with small, single-finger movements performed while the other fingers remain still. The paper uses data-driven analysis of everyday hand-object actions to identify gestures that are distinguishable from natural movement, then evaluates their robustness across objects and actions.

How the study was done

  • 7,488 gesture trials
  • 933 everyday-action trials
  • 36 objects evaluated

The work combines user studies and data-driven analysis. The evaluation includes 7,488 intentional gesture trials and 933 trials of everyday hand-object actions across 36 objects. The authors first use an interpretable threshold-based classifier to analyze false activations, then demonstrate a proof-of-concept system with a commercially available VR glove and a random-forest classifier.

Key findings

  1. Core idea

    Move one finger while the others stay still

    Single-finger movement stands out because everyday grasping and object manipulation more often involve static fingers or several fingers moving together.

  2. Robustness

    High precision across varied objects

    The threshold-based recognizer achieved 100% average precision and 88% average recall across three primitive gestures performed on 36 objects.

  3. False activations

    Most everyday actions did not trigger input

    Only 51 of 933 everyday-action trials caused false activations; 23 actions produced none, while very small or deformable objects accounted for most errors.

  4. Prototype

    The approach worked on commodity hardware

    The VR-glove prototype reached 86% accuracy with 17 false activations in 800 trials; when the held object was known, accuracy rose to 89% with no false activations in the collected dataset.

What should change

Microgesture designers can reduce unintended input by choosing movements that are naturally uncommon during the primary task. Single-finger gestures offer a simple alternative to complex gesture sequences or separate delimiter actions, and can support interaction with a broad range of handheld objects.

Abstract

SoloFinger is a microgesture technique for interacting while grasping everyday objects. It uses intentional movement of one finger while the remaining fingers stay idle, taking advantage of the fact that this pattern is uncommon during natural hand-object actions. The authors evaluate the approach against a dataset of everyday actions and demonstrate implementations using interpretable threshold-based recognition and a random-forest classifier. Results show that the technique can support convenient input across varied objects while keeping unintended activations low.

Cite this paper