Synthesis of Human-in-the-Loop Navigational Operations towards Maritime Autonomous Surface Ships
Abstract
Maritime transportation is indispensable to the world economy as it dominates over 95 % of the trade volume, so stakeholders have been striving to promote maritime transportation efficiency and sailing security. Benefitted from the technical progress of advanced guidance, navigation, and control techniques, industrial digitalization, and artificial intelligence, the concept of maritime autonomous surface ships (MASS) has emerged. However, according to the regulatory scoping exercise by the International Maritime Organization, several critical phases with different degrees of autonomy are prerequisites to achieving full ship autonomy. In these phases, human practitioners participate the navigation operations loop at various levels in terms of intervention and operating venues, which means human navigators will stay in the loop until the final phase - fully autonomous ships - comes. Though, as professionals, human navigators have recorded numerous safe sailings and accumulated rich experience on the ship bridge, the major cause of most marine accidents is still attributed to human factors. In this regard, the research on human-in-the-loop (HITL) navigational operations will have impact on not only the enhancement of marine traffic safety, especially in the period when human navigators still play the dominant roles on the ship bridge; but also on the development of MASS, as humans contribute both expert knowledge and faulty cases onboard and in complex marine traffic systems as the input of the ship intelligence.
The study of synthesis of HITL navigational operations is thus motivated and proposed to address the human-related issues towards the development of MASS by integration of maximizing expertise knowledge, monitoring on-bridge operations, summarizing human navigational logics and modeling the mechanism, and providing practical decision support tools. In establishing such a synthesis study framework, experimental facilities are based on different maritime ship-bridge simulators, while techniques involved in the route map can be categorized into three groups in terms of the aim of utilization: on-bridge monitoring & data collection (e.g., sensor fusion, computer vision, and motion/gesture/eye-movement tracking), analysis and learning of operational behaviors (e.g., statistics, pattern recognition, and expert system), and online surveillance & decision support (e.g., situation awareness, risk management, and collision avoidance algorithms). The experimental platforms incorporate the techniques in the route map and then become the rudiment of intelligent ship bridges on MASS.
This dissertation explores the synthesis of HITL navigational operations towards MASS, especially in the experimental design & implementation and HITL applications. The techniques in the route map are adapted and applied in different scenarios based on various platforms. Three case studies are conducted to demonstrate how the synthesis study framework can be carried out to comprehend HITL navigational operations. The first relates to expertise-knowledge-aided path routing to increase sailing safety; the second conceptualizes navigating patterns analysis and illustrates a built-on decision support system for collision avoidance; the third discusses the navigational visual attention and improves the measurement solution. The case studies validate the applicability of the synthesis study framework.
Has parts
Paper 1: Wu, Baiheng; Li, Guoyuan; Zhao, Luman; Hildre, Hans Petter; Zhang, Houxiang. A human-expertise based statistical method for analysis of log data from a commuter ferry. I: Proceedings of the 15th IEEE Conference on Industrial Electronics and Applications(ICIEA 2020)Paper 2: Wu, Baiheng; Li, Guoyuan; Wang, Tongtong; Hildre, Hans Petter; Zhang, Houxiang. Sailing status recognition to enhance safety awareness and path routing for a commuter ferry. Ships and Offshore Structures 2021 ;Volum 16. s. 1-12 https://doi.org/10.1080/17445302.2021.1907084 This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (CC BY-NC-ND 4.0)
Paper 3: Wu, Baiheng; Li, Guoyuan; Zhao, Luman; Aandahl, Hans-Ingar Johansen; Hildre, Hans Petter; Zhang, Houxiang. Navigating Patterns Analysis for Onboard Guidance Support in Crossing Collision-Avoidance Operations. IEEE Intelligent Transportation Systems Magazine (ITSM) 2021 ;Volum 14.(3) s. 62-77 https//doi.org/10.1109/MITS.2021.3108473
Paper 4: Wu, B., Zhao, L., Thattavelil Sunilkumar, S. R., Hildre, H. P., Zhang, H., & Li, G. Visual Attention Analysis for Critical Operations in Maritime Collision Avoidance
Paper 5: Wu, B., Han, P., Hildre, H. P., Zhao, L., Zhang, H., & Li, G. A Camera-based Deep-Learning Solution for Visual Attention Zone Recognition in Maritime Navigational Operations
Paper 6: Wu, B., Zhao, L., Hildre, H. P., Zhang, H., & Li, G. Evaluation on Effectiveness of Electronic Chart System for Maritime Navigators Based on Visual Attention and Risk Assessment
Paper 7: Wu, B., Sæter, M.L., Hildre, H. P., Zhang, H., & Li, G. Experiment Design and Implementation for Human-in-the-Loop Study Towards Maritime Autonomous Surface Ships