Intelligent low-cost solutions for underwater intervention using computer vision and machine learning

Abstract

This thesis considers intelligent solutions that facilitates for autonomous technology in underwater intervention and navigation. A special focus have been on implementing methods and solutions in inspection, maintenance, and repair (IMR) operations using low cost equipment. The presented work involves development and implementation of solutions to increase the efficiency and safety of operations, and includes both theoretical contributions and experimental testing. The work includes learning algorithms to improve visual authenticity of simulators and digital twin scenarios, computer vision in guidance and navigation of underwater vehicles and intervention systems, development and testing of novel equipment, and experimental verification of presented methods and equipment.

The introduction of advanced learning algorithms enables systems to perform tasks that was previous too complex and complicated for any modelled solutions. This thesis explores the use of generative adversarial networks to improve the realism of simulated environments, which again will improve the transferred learning between simulated environments and real world operations. Such a mapping between domains is complex to model, especially in the underwater scene given the intricate scenery with scattering of light and marine particles. Machine learning algorithms provides new solutions for this mapping, and can aid in improving result from simulation tools to have greater impact on real world operations.

In the same way humans uses their senses to experience life, autonomous systems requires sensory feedback to act and react upon. A sensor is only as effective as the information that can be extracted from the sensory output, and increasing and strengthening this information will improve the support from the sensor. Camera footage contain information with higher spatial and temporal resolution than acoustic information, and utilizing this information to its full will improve today's sensory systems. This thesis explores the use of visual aid and the potential it brings to increase autonomous capabilities of underwater vehicles and intervention systems. The explored methods includes object detection and tracking networks trained on custom dataset that are able to locate objects within the camera frame. Robust localization of objects is vital in intervention operations to perform autonomous grasping efficiently and safe.

The area of focus for this thesis has been the research and development of low cost solutions for intervention using machine learning. The platform for testing has been the SeaArm-2 manipulator, which is a small electric modular manipulator with an integrated end-effector camera. Part of the thesis work has included development and assembly of the manipulator which is designed to enable both remote control and is optimized for autonomous operations, where the camera enables visual aid to be used as the main sensor input in autonomous grasping operations. Object detectors and trackers in combination with mathematical 3D feature extractors provides a robust system capable of locating the relative 3D position of objects, or features, of interest. A large part of this thesis is dedicated to experimental verification of methods using deep learning and visual aid. This is implemented on the SeaArm-2 manipulator and an underwater vehicle for experimental testing of autonomous functionalities. Experimental testing is important in order to verify the proposed methods and solutions, and in combination with improving results from simulations, and development of digital twins, this thesis provides a good platform for advancing towards a more autonomous tomorrow.