Internal Models as Echo State Networks: Learning to Execute Arm Movements

Abstract

As robots are becoming more and more complex, with higher degrees-of-freedom, lighter limbs, and springy joints, it becomes harder to control their movements. New approaches, inspired from neuroscience, are attracting increased attention among computer scientists dealing with motor control.

The focus in this thesis is on how robots can learn to control their limbs by learning how their bodies work, i.e., by learning internal models of their motor apparatus. Inspiration from cerebellar research combined with concepts from traditional control theory has been used as a basis.

The research in the thesis is twofold. First, we investigate how internal models can be used to solve different control problems. In particular, we consider how to handle delays in the sensory-motor-loop and how to realize bimanual coordination. Second, we study how the internal models can be represented and learned. This includes how to choose movements to learn from in order to learn as much of the internal model as possible and how to actually learn the training movement.

A simple simulator is used in the experiments and the simulator’s internal models were implemented as echo state networks (ESNs), a relatively new and promising type of recurrent neural networks. The simulator learns internal modes of his motor apparatus by imitating human motion. Human motion data was recorded and the task of the simulator’s control system is to generate motor commands that result in the simulator replicating the recorded movement.

From the experiments we conclude that using ESNs for representing and learning internal models looks promising. With an ESN we are able to generalize to imitating novel movements, and we demonstrate that it is able to learn various bimanual coordination patterns. However, training ESNs is challenging and a major contribution from this thesis is a novel training method that works particularly well in our application. The thesis also contributes to how different internal models can be used and trained together.