iCub humanoid robot learns language like a toddler

According to new results from researchers at the University of Hertfordshire, robots can develop basic language skills through interaction with a human. The research is a part of iTalk project, where previously described iCub robot is used as a platform which listens to humans in a similar way like human babies of about 6 to 14 months when they transition from babbling to first word forms.

Dr Caroline Lyon, Professor Chrystopher Nehaniv and Dr Joe Saunders have carried out experiments as a part of the iTalk project to show how language learning emerges. They used a humanoid iCub robot named DeeChee which could initially only babble and perceive speech as a string of sounds, rater than divided words.

Participants are shown the experimental set up of words related to shapes and colors they are asked to teach DeeChee. After engaging in a few minutes of “conversation” with humans, in which the participants were instructed to speak to the robot as if it were a small child, the robot adapted its output to the most frequently heard syllables to produce some word forms such as the names of simple shapes and colors.

“It is known that infants are sensitive to the frequency of sounds in speech, and these experiments show how this sensitivity can be modeled and contribute to the learning of word forms by a robot”, said Lyon.

DeeChee interprets the speech from the participant as a stream of phonemes, from which all possible syllables are formed and stored. The performance of phoneme recognizers is hard to assess, since results varied a lot. Since the participants weren’t told to use the same teaching set of sentences, there is a wide range in the total number of different syllables interpreted by the robot for each participant and it ranged from 52 to 549.

Although the iCub robot is learning to produce word forms, it does not know their meaning, and learning meanings is another part of the iTalk project’s research.  These scientific and technological advances could have a significant impact on the future generation of interactive robotic systems.

For more information about the used methods, read the full paper published in PLoS ONE: “Interactive Language Learning by Robots: The Transition from Babbling to Word Forms”.