Mouseless – an invisible computer mouse

Pranav Mistry, Liyan Chang, and Pattie Maes, from the Fluid Interfaces Group at the MIT Media Lab, were interested in removing the requirement for a physical mouse, while still allowing computer users to interact with their computers in a way with which they are familiar with. They developed an “invisible mouse” and named it Mouseless. If the names of the researchers sound familiar, it is because we already wrote several articles about their projects as SixthSense or Siftables.

Since the computer mouse didn’t suffer any extreme changes over the last couple of decades, and there are various multitouch and gestural interaction technologies available today, it is expected that new information interaction technologies will emerge in near future. The folks from Fluid Interfaces Group were thinking of an easy way for transition where users wouldn’t have to be trained to use the new information interaction technology and came up with Mouseless.

The Mouseless invention removes the requirement of having a physical mouse altogether but still provides the intuitive interaction of a physical mouse that most of us are familiar with. Mouseless consists of an Infrared (IR) laser beam (with line cap) and an Infrared camera. Both IR laser and IR camera are embedded in the computer. The laser beam module is modified with a line cap and placed in a way so it creates a plane of IR laser just above the surface the computer sits on. The user cups their hand, as if a physical mouse was present underneath, and the laser beam lights up the hand which is in contact with the surface. The IR camera detects those bright IR blobs by using computer vision.

The change in the position and arrangements of these blobs are interpreted as mouse cursor movement and mouse clicks. As the user moves their hand the cursor on screen moves accordingly. When the user taps their index finger, the size of the blob changes and the camera recognizes the intended mouse click.

As they improve their computer vision algorithms, an extensive library of gestures could be implemented in addition to mouse movement and mouse clicks. Typical multitouch gestures, such as zooming in and out, as well as novel gestures, such as balling one’s fist are all possible. The researchers claim that the use of multiple laser beams would allow the recognition of a wider range of free hand motions, enabling novel gestures that the hardware mouse cannot support.

There are no plans for commercializing the “invisible mouse” but the prototype Mouseless was built for around $20 USD. Although it is not expensive to realize, it offers advanced gestural usage, and its usage is intuitive to all who have used an ordinary computer mouse, there are still some issues that need to be solved. Aside the development of the algorithms for better gesture recognition, there is a question regarding the long-term use, as well as the fact that you would have to be fixed to the spot where the IR rig is set in order to use this technology.