Identification of Failure Modes in the Collision Avoidance System of an Autonomous Ferry using Adaptive Stress Testing
Peer reviewed, Journal article
Published version
Date
2022Metadata
Show full item recordCollections
Abstract
As complex autonomous systems emerge in the maritime sector, measures must be taken in order to ensure thorough safety assessment. Real world testing can be costly and potentially dangerous, and therefore there is a need for suitable simulation-based methods. This paper presents an implementation of the Adaptive Stress Testing (AST) method applied to the collision avoidance (COLAV) system of a small passenger ferry. AST is a simulation-based technique which has shown promising results in safety assessment of aviation and automobile systems. Given a simulator of a system, AST uses reinforcement learning to optimize towards system failure, and returns the most likely failure scenarios. AST is here shown to successfully identify scenarios where the criteria for failure are met, which is when the ferry collides with an adversary vessel controlled by AST. However, the majority of the initial results exhibit failures where the COLAV system of the ferry is not responsible for the failure, which makes the results less valuable to system developers. In order to improve the relevance, augmentations are made to the optimization problem. The augmentations result in four distinct problem formulations which are presented in the paper. Finally, the results are clustered using an unsupervised machine learning method called Soft Dynamic Time Warping k-means clustering in order to present a general summary of the identified failure scenarios. Our results demonstrate the relevance and potential of AST for the maritime sector, and illustrates how common drawbacks of the AST method can be circumvented by method adjustment. Identification of Failure Modes in the Collision Avoidance System of an Autonomous Ferry using Adaptive Stress Testing