Hellinger Metrics for Validating High Fidelity Simulators Using Target Tracking

Abstract

To achieve autonomy at sea, it is believed simulations will be essential in testing, analysing and verifying autonomous systems due to the scarcity and high cost of obtaining real data for all relevant scenarios. This reliance on simulation raises the question on how much synthetic data can be trusted, especially from sensor data such as lidars, radars and cameras. Methods for validating specific sensor models exists, however these are often focusing on perceptional differences without considering the sensors impact on the autonomy’s situational awareness. In this paper we make an attempt to analyse this using a JIPDA target tracker, by comparing its performance on real and synthetic lidar data with various Hellinger metrics for Bernoulli and multi-Bernoulli multi-target densities. Our work showcases a method that quantifies sensor fidelity of synthetic data based on tracker performance, a step towards building trust to simulators targeted at validating autonomy systems.

Hellinger Metrics for Validating High Fidelity Simulators Using Target Tracking