Exploring a Developmental Reservoir Computing System Using Self-Modifying Recurrent Cartesian Genetic Programming

Abstract

Inspired by biology, numerous new computational models have been proposed as alternatives to cope with the ever-growing complexity of the traditional von Neumann architecture. Vastly parallel systems comprising simple units that only interact locally, form the basis of many of those new systems.

In this thesis, we combine ideas proposed in the field of bio-inspired unconventional architectures. Specifically, we explore the possibility of evolving a developmental reservoir. The reservoir is the heart of a computational model, coined reservoir computing. A reservoir computing system works by perturbing the reservoir with a stream of data. The reservoir will extract high-dimensional features of the data stream, which is classified in a readout layer by a linear classifier. A static network with a random recurrent topology is often used as a reservoir. However, we propose a self-modifying reservoir that is able to develop and adapt to the perturbations, such that the reservoir structure self-organises in a way that enables it to transform the input into a high-dimensional feature set. Additionally, development will enable a large reservoir to be grown from a relatively small genotype.

The system implemented is an extension of a recurrent Cartesian genetic programming reservoir computing system presented in the specialisation project by the author. The extension support the self-modifying operations required in a developmental system.

Fitness functions based on separability and development is used in the endeavour of finding a self-organising computationally capable reservoir. We will explore how the of genotype size, and the environment affect the reservoir. Additionally, the temporal parity problem is solved to demonstrate the system's performance.

The results show that finding a genotype that develops into a reservoir with the aforementioned features is rather difficult. Nevertheless, examples of working genotypes are found, serving as a proof of concept.