iCub robot with simplified artificial brain learns languages

Two researchers in France, affiliated with Inserm U846 Stem Cell and Brain Research Institute and Université Claude Bernard Lyon I, used iCub humanoid robot as a platform to develop a simplified artificial brain system which enables the robot to learn and understand sentences containing a new grammatical structure, as well as to predict the end of a sentence.

One of the most remarkable aspects of language-processing is the speed at which it is performed. For example, the human brain processes the first words of a sentence in real time and anticipates what follows, thus improving the speed with which humans process information. Still in real time, the brain continually revises its predictions through interaction between new information and a previously created context.

Dr Xavier Hinaut and Dr Peter Ford Dominey used that ability as inspiration to develop an artificial brain that uses a “neuronal construction” similar to one used by human brain. Thanks to what researchers refer to as recurrent construction this artificial brain system can understand new sentences with new grammatical structure. Recurrent construction relies on connections that create locally recurring loops which link two sentences and can even predict the end of a sentence before it is provided.

As you can see in the following video demonstration, when a researcher asks the robot to point to a specific object and move another specific object, the robot repeats the sentence and explains that it has fully understood what it has been asked to do before it takes actions.

Although actions executions are somewhat slow, the system can be used to explore the way our brains processes language. This research has another important implication, that of contributing to the ability of robots to learn a language one day.

“We know that when an unexpected word occurs in a sentence, the brain reacts in a particular way. These reactions could hitherto be recorded by sensors placed on the scalp”, said Dominey, Director of Research at Inserm U846.

The model enables identification of the source of these responses in the brain. If this model is accurate, it could enable treatment of various linguistic malfunctions. For instance, it could contribute to possible linguistic malfunctions in Parkinson’s disease.

“At present, engineers are simply unable to program all of the knowledge required in a robot. We now know that the way in which robots acquire their knowledge of the world could be partially achieved through a learning process – in the same way as children”, said Dominey.

For more information, you can read the article published in PLOS ONE: “Real-Time Parallel Processing of Grammatical Structure in the Fronto-Striatal System: A Recurrent Network Simulation Study Using Reservoir Computing”.