WorldKit – touch-based interfaces on everyday surfaces

In one of our previous articles we described OmniTouch – a prototype of a wearable multitouch system that turns any surface into interface for information interaction. Researchers at the Carnegie Mellon University (CMU) have played with that idea and created WorldKit – a system able to create these touch-based interfaces and enable information interaction in our living or working spaces.

CMU’s WorldKit system relies on a ceiling-mounted camera and projector to record room geometries, sense hand gestures and project images on desired surfaces. The system enables users to rub the arm of a sofa to “paint” a remote control for appliances or swipe a hand across an office door to draw a calendar with notes about obligations. These interfaces can be moved, modified or deleted with similar gestures.

“Depth sensors are getting better and projectors just keep getting smaller”, said Robert Xiao, a doctoral student at CMU’s Human-Computer Interaction Institute (HCII). “We envision an interactive ‘light bulb’ – a miniaturized device that could be screwed into an ordinary light fixture and pointed or moved to wherever an interface is needed.”

Xiao developed WorldKit with Scott Hudson, an HCII professor, and Chris Harrison, a Ph.D. student. The system does not require prior calibration and it automatically adjusts its sensing and image projection to the orientation of the chosen surface. Users can use gestures to summon switches, message boards, indicator lights and a variety of other interface designs from a menu. Ultimately, the WorldKit team anticipates that users will be able to create their own custom design interfaces with gestures.

“People have talked about creating smart environments, where sensors, displays and computers are interwoven”, said Harrison. “But usually, that doesn’t amount to much besides mounting a camera up on the ceiling. The room may be smart, but it has no outlet for that smartness. With WorldKit, we say forget touchscreens and go straight to projectors, which can make the room truly interactive.”

Though WorldKit now focuses on interacting with surfaces, CMU researchers envision that their system could one day be used to interact with projections in free space. Likewise, higher resolution depth cameras and improved recognition methods could someday enable the system to sense detailed finger gestures. In addition to gestures, the system also could be designed to respond to voice commands.