Visual Analytics for Anomaly Detection in Maritime Big Data

Abstract

With the increase of the amount of collected data during each year, comes the need to process and understand it. Maritime domain is steadily integrating more sensors into the ship systems to enable health monitoring, help with onboard decision-making and facilitate better-optimized engineering approaches. The data can be turned into insight by domain experts with the help of visual analytics. This thesis describes ideas and processes behind the development of visual analytics framework for a maritime sensor dataset and its application in detecting anomalies in the data, using propeller ventilation detection as the domain problem. The framework is based on principles of effective data visualization, visual exploration, knowledge generation, feature engineering and machine learning. Due to the difficulty of the domain problem, domain experts were integrated into visualization-assisted feature engineering improvement loop. A prototype was developed with an interactive interface, multiple visualization tools and a backend data processing pipeline. The pipeline extracts features from time series and prepares the data for clustering. The clustering separates the recordings into distinct sets, which, with the help of domain experts can be labeled and identified. The framework includes both interactive visualizations for deep inspection of relevant records and static visualizations, which can be generated ahead of time, for higher-level overview and evaluation. The framework was tested on ventilation detection problem and improved with the help of domain expert involvement.

Keywords: visual analytics, data visualization, anomaly detection, feature engineering, maritime operations.