Cyber-resilient Aided Inertial Navigation: Applications to Ships and Unmanned Surface Vehicles

Abstract

With the recent advancement in maritime autonomy, the autonomous ship market is predicted to cut costs through more efficient operation of uncrewed ships and surface vehicles, followed by a reduction in associated construction and maintenance costs. However, without human fallback, the safety of autonomous ships is solely dependent on the sensors and computerized systems onboard the ship. In case of sensor failure or degradation, this may cause fatal consequences, especially in safety-critical operations such as ship maneuvering and docking in urban environments that require highly accurate and robust navigation signals to operate safely. To counteract this concern, we seek to improve the safety of autonomous ships by developing highly robust and redundant navigation systems applicable to small unmanned surface vehicles (USVs) in urban environments in this thesis. Since Global Navigation Satellite Systems (GNSS) can be unreliable and inaccurate in urban environments, other sensors should be used to develop independent, GNSS-free navigation solutions for increased redundancy and reliability. In the first step, we establish a set of artificial landmarks as the main method to aid the vehicle in urban environments. They are used to estimate the relative pose between the camera onboard the vehicle and the landmarks at the dockside. They are advantageous to use since they have a distinct pattern that is significantly easier to find in the visual scene compared to naturally occurring features, especially in unstructured and complex outdoor environments. In addition, such landmarks can be used to produce drift-free camera-tag pose measurements and therefore function as a local navigation alternative to GNSS if the target point is known in a global frame. Initially, a learning-based detector was used to recognize the proposed landmarks, also referred to as visual fiducial tags, and showed superior performance compared to a conventional model-based detector, given the contemporary design of the tag configuration. However, after conducting several improvements to the tag configuration, the learning-based detector was replaced with a lightweight yet robust model-based detector to reduce model complexity and enhance model interpretation, which are important properties in safety-critical operations. Next, we extend the representation of weather data by providing a rich dataset containing normal weather conditions and conditions with reduced visibility, such as rain, sunlight, darkness, and fog, to test the robustness of the proposed visual fiducial system. The visual fiducial system consists of a detector, a coding system, and a pose estimation algorithm that is used to estimate the camera-tag pose from the corners of the tags. Except for extreme scenarios in which the visibility was significantly reduced due to, for example, complete darkness or blurry and distorted camera lens, the visual fiducial system proved to work very well. Next, we look into state estimation since the full state of the vehicle is necessary for closed-loop motion control. Hence, we develop, implement, and tune a novel error-state Kalman Filter for robust state estimation of the vehicle by fusing camera-tag pose measurements coming from the visual fiducial system and inertial measurements coming from the inertial measurement unit (IMU). We also derive and employ transformations between the coordinate frames necessary for the sensor fusion scheme. Finally, we demonstrate that the proposed method performs well for closed-loop docking of a USV, independent of GNSS aiding, through _eld experiments in the harbor. In other operations, such as harbor maneuvering, the vehicle will not always be close to landmarks. As a result, the state estimates driven by IMU measurements will quickly drift when the landmarks are out of range. To extend the scope of operation of the vehicle, we equip the vehicle with a Doppler velocity log (DVL) which provides low-drift velocity aiding. Experimental results show that the proposed landmark-based and acoustic-based navigation systems complement each other, which enables more flexible and long-endurance operations. For example, the DVL allows the vehicle to maintain a low-drift state estimate for a period, typically a couple of minutes, without the need for absolute aiding from, e.g., landmarks or GNSS. The visual fiducial system can then be used to eliminate the induced drift when the vehicle is close to landmarks. Although robust, GNSS-free navigation solutions applicable to small USVs are developed in this thesis, the transmission of associated sensor and navigation signals present attack surfaces that adversaries may exploit. Targeting bandwidth-greedy sensors such as cameras and LiDARs, typically used in many navigation and perception tasks, we are interested in securing the associated sensor data without comprimising performance to avoid large time delays in the feedback loop. In this context, dedicated computers are often used for data acquisition of such sensors in distributed guidance, navigation, and control (GNC) systems. However, distributing such sensor data to the different components of the GNC system is inherently insecure, and adversaries are free to perform cyber-physical attacks such as eavesdropping and injection attacks. To resist such attacks without inducing large time delays, we implement and benchmark high-performance stream ciphers and authenticated encryption algorithms onboard embedded devices and suggest using these algorithms instead of block ciphers. This is because the keystream generation phase of a stream cipher is much more efficient than that of a block cipher. Concerning computational efficiency, we also develop a “compress-then-encrypt” scheme that is used on image data and conclude that compression should only be employed if the network bandwidth is constrained. We also look into the perspective of an adversary. As a basis, we use a distributed GNC architecture where the navigation system and the guidance & control system run on separate computational devices, which makes the resulting closed-loop signals vulnerable to cyber-physical attacks if not properly secured. By spoofing the address resolution protocol using a Man-in-The-Middle computer, more sophisticated attacks, such as manipulation of position and heading estimates of a USV, were developed before the actual effects were demonstrated in the _eld. Finally, we develop a cryptographic scheme that combines authenticated encryption with timestamps to resist both injection and replay attacks and show through _eld experiments that the proposed cryptographic scheme effectively secures the USV against such attacks.