Musical robot companion Shimi is more than a docking system

Georgia Tech’s Center for Music Technology developed a robot whose role is to be a musical companion. Named Shimi, the robotic musical companion is able to recommend songs, dance to the beat, and even create new, unique musical compositions. Instead being just a docking station, the robot increases interactivity and various features which can be introduced with smartphone apps.

“Shimi is designed to change the way that people enjoy and think about their music”, said Professor Gil Weinberg, director of Georgia Tech’s Center for Music Technology and the robot’s creator. “If robots are going to arrive in homes, we think that they will be these kind of machines – small, entertaining and fun. They will enhance your life and pave the way for more sophisticated service robots in our lives.”

Shimi acts as a robotic docking station with a “brain” powered by an Android or iPhone smartphone. Once the smartphone is docked, the robot recognizes the type of it and relies on sensing and musical playback capabilities of the user’s mobile device in order to stream the music from the device onto the hi-def speakers which are positioned as the robot’s ears.

Shimi can interact with its users by relying on sensors of the smartphone. By employing the video data from smartphone’s camera, it can use facial and speech recognition algorithm to follow a listener around the room and position its speakers in right direction to provide the optimal sound. While this feature is useful for a single user, it could be a nuisance with multiple people around the robot.

Another recognition feature is based on rhythm and tempo detection. If the user taps or claps a beat, Shimi analyzes it, scans the phone’s musical library and immediately plays the song that best matches its ability to interpret the suggestion. This detection also allows the robot to dance to the rhythm.

“Many people think that robots are limited by their programming instructions”, said Music Technology Ph.D. candidate Mason Bretan. “Shimi shows us that robots can be creative and interactive.”

Bad news if you tend to perform inarticulate dancing, since Shimi’s developers work on its gesture recognition. They plan to develop features which will make the robot more interactive. New features will include the ability to detect whether the users are shaking their head in disagreement or wave a hand in the air to signal Shimi to skip to the next song or adjust the volume.

Once integrated with other apps on your smartphone, Shimi could prove useful by helping you with notifications about events in your online social life by providing sound or gesture feedback. For instance, the robot could alert you when you get a Facebook post, do a shimmy when you get a new Twitter follower, or broadcast your dance parties to your remote friends around the world.

The robot will also have the capability to recommend new music based on the user’s song choices and provide feedback on the music play list, and the researchers hope that other developers will come with more apps and games that will improve Shimi’s creative and interactive capabilities.

In order to commercialize the idea, a group of robot researchers and entrepreneurs from Georgia Tech, IDC in Israel, and MIT Media Lab, formed a company named Tovbot which will make the robot available to consumers by the 2013 holiday season. Its price hasn’t been announced yet.