| dc.description.abstract | In this report we have looked at the application of the evolutionary process applied to neural networks, and in particular the CTRNN network type. We have experimented with the network structure, network parameters and all evolutionary parameters in order to find which combinations provides the most effective solution for our problem. The given task was to find a suitable problem for the E-Puck and apply these techniques to evolve a controller able to complete the goal. By adapting existing solutions we created a test environment using Breve and Pucker and set up a virtual environment corresponding to what we stated in the problem specification. The CTRNN provides multiple ways of creating memory using time constant, recurrence and self-connections, but to be certain that no shortcuts were taken by our evolved prototype controller that used purely reactive behavior, we added measures to avoid this by amongst other adding the "reposition zone". Although attempts were made we did not succeed in finding good solutions to run fitness tests in parallel and thus we were stuck with quite long simulation times, for most experiments several days. This somewhat limited the degree at which we could experiment with and tune all the net- and evolutionary parameters. A big challenge was to set up the fitness tests and fitness functions in such a way that it provided a smooth progression through the fitness landscape. When placing the E-Puck a bit back from the junction the first evolutionary steps goes in the same direction for both goals. In the T-junction there are however two conflicting goals which each must be given a fitness. Improvements in achieving one goal (Like turning left a lot) can come at the expense of the other goal. If one goal is easier to improve on than the other it can lead to advances in the second goal being ignored and these individuals being replaced by those performing much better in the easy to obtain goal. For our experiment the first steps in the evolution of a split in behavior based on the light signals given seemed to be the most difficult, and the more training was done before this problem arose (Time spent getting the candidates to the junction) the harder it was to evolve. By starting the earliest fitness tests with the E-Puck close to the junction we exposed the prototype to the problem early and got the desired behavior pretty quick. The downside of course was that the entire network needed to be gradually adjusted away from dealing with that situation and still keep these properties. This chosen solution of gradually moving the starting location is by all likelihood the main reason why the final candidates are indeed starting to slightly turn from the start of the runs. The evolved behavior is dominated by wall avoidance behavior coupled with a biased tendency to turn in one direction, depending on the initial light signal, as opposed to recognizing the junction and coupling it with the earlier observed signal. The structure of the neural network has remained fairly constant through our experiments. We never changed the 11 nodes in the input layer. We quickly decided that the best solution was to only have one output neuron, and so we experimented a bit with what was in between. Without the connections between the nodes within the same layer that was added from experiment 3 the room of possible circuits is quite small. When we made this layer fully connected it allowed for potentially a lot more complex circuits however. We did not observe big changes in behavior nor the speed of evolution after this, except for a slight increase in iterations needed. The end result was positive in the way that we managed to evolve a prototype controller that managed to reach the proximity of both goals depending on the light signal given. As can be expected with evolution the solution found was not working exactly as what we had in mind when we started the task since it ended up being more of a wall following robot with the light signal biasing which wall to follow. Even with the anti-wall-following trick of reseting the position back to the center of the corridor the solution still became somewhat the same. Our experiments were conducted without deliberate noisy information. In other experiments using the pucker librarycite{kjell} where the goal was to apply the solution to real world robots the transition was not smooth. Due to the large amount of approximations made in the Pucker library like friction, exact motor speeds, motor power, sensory locations and sensory inputs to name a few it is very unlikely our controller would have been able to succeed in the same task had it been moved to a physical E-Puck. We saw that by only changing the starting direction of our E-Puck in the simulation by some degrees the best evolved E-Puck ran into the wall and got stuck. We therefor have to conclude that our evolved prototype is fragile at best, and not suitable for physical implementation. It is however quite possible that given more time and more iterations one could make the solution more robust and dynamic by giving a bit more noisy data, slightly varied starting locations and rotations, and by varying the effective motor speed. In this project we have shown that evolution is a powerful and dynamic tool. The exact task we worked with in this project could have been solved with classic engineering, but we have little doubt that in more dynamic situations (No straight walls for one) evolution can be an efficient and in many ways superior strategy for creating controllers. The CTRNN has proved to be a very dynamic tool used in combination with evolution and getting these to work has been a inspiring exercise for us. In a broader scientific setting we have not achieved anything new, but this project has given a taste of what can be possible, and it is surely something we will use again in the future should an occasion present itself. | nb_NO |
