On the Piecewise Affine Representation of Neural Networks

Abstract

Det kan vises at nevrale nett kan uttrykkes som stykkevise affine (PWA) funksjoner. Men, forskning har fokusert på å telle de lineære regionene, fremfor å finne den eksplitte PWA formen. Denne oppgaven fremfører en algoritme som finner PWA formen til nevrale nett, og bruker dette til å utforske applikasjoner innen modellering og reguleringsteknikk.

In exchange for large quantities of data and processing power, learning algorithms have yielded models that provide state of the art predication capabilities in many fields. However, the lack of strong guarantees on their behaviour have raised concerns over their use in safety-critical applications. It can be shown that neural networks with piecewise affine (PWA) activation functions are themselves PWA, with their domains consisting of a vast number of linear regions. Research on this topic has focused on counting the number of linear regions, rather than obtaining the explicit PWA representation. This thesis presents a novel algorithm that can compute the PWA form of fully connected neural networks with ReLU activations. Several case studies regarding the usefulness of this representation in terms of modeling and control are undertaken. Nominal stability results for a simple dynamical system based on a small neural network are obtained via the Lyapunov method for PWA systems, and suggestions for extending the approach to neural networks of arbitrary size are outlined. Moreover, the practicality of using MPC and data-driven methods to control neural networks is investigated.