Semantic Segmentation of Radar Data with Deep Learning

Abstract

Autonomous ships have the potential to redefine the maritime industry, providing substantial improvements in safety, economics and fuel efficiency, while creating new previously unimagined services. Radar is a key part of the solution to the navigation and collision avoidance challenge for autonomous vessels, due to its high range and accuracy, as well as robustness against harsh weather conditions. A semantic segmentation simplifies the radar data into homogeneous clusters of defined targets of interest, allowing the autonomous agent to understand its environment at a resolution suitable for high-level reasoning and planning.

A large database of maritime radar envelope data was gathered from two S-band radar systems along the Norwegian coastline, and ground truth labels were constructed from Automatic Identification System (AIS) data and chart data. A Machine Learning (ML) detector, in the form of a Convolutional Neural Network (CNN), was trained in a supervised manner to segment the radar envelope image into four classes: background (sea), vessels, land and unknown (i.e. regions that the radar s signals can not reach). Several modifications to CNN architectures and augmentations to radar envelope data are suggested and evaluated, with downsampling of the envelope data increasing the receptive field of the CNN units and replacing batch normalization layers with group normalization layers, alleviating the adverse effects of limited memory, having the most significant contributions.

The ML detector exhibits great results on the test set: correctly classifying over 99% of the navigable area, an overall 69.11 mean Intersection Over Union (IoU) score and a mean of 75.55 on the Boundary Jaccard (BJ) index. The vessel detection capability is compared to a traditional approach to radar detection in the form of a Constant False Alarm Rate (CFAR) detector supplied by Kongsberg Seatex. The ML detector achieves a higher probability of detection for all false alarm rates, with both detectors managing a 0.9 probability of detection for a false alarm rate around 2 · 10−2. Below this point, the ML detector outperforms the CFAR detector by several orders of magnitude on the false alarm rate. The ML detector also exhibits superior streak performance; the average and maximum period that vessels go undetected is significantly shorter, especially at lower false alarm rates. This performance discrepancy can in part be explained by lower-than-expected performance from a CFAR detector, owing to a difficult, multimodal data distribution and issues with the vessel labels.

The master project and all experiments are conducted on regular consumer hardware. The ML detector processes data at a fraction of the data collection rate of radar systems (0.3 vs. 2.4 seconds). The developed detector system could therefore be deployed as is, but more research on the effects of adversarial techniques should be conducted before it is employed for safety-critical tasks.