| dc.description.abstract | In recent years there has been a drastic increase in the commercial availability of
small UAVs for civil and commercial applications. These applications includes,
but are not limited to, inspections of structures, search and rescue, monitoring and
surveillance of oil spills and autonomous border control. All of these applications
emphasize the UAV’s role as a remote sensing platform with the main focus on
visual sensors and data collection, and also includes an object detection and tracking
component. Object detection and tracking in UAVs using visual sensors can
generally be divided into two subproblems, where the first problem is that of controlling
the path and orientation of the UAV in such a way that the UAV is able
to capture useful data with its on-board sensors. The second problem is that of
making the UAV able to automatically detect and track objects of interest using
the UAV’s on-board sensors and autopilot. The former subproblem is considered
a problem of path planning, while the second subproblem is considered a problem
of machine vision.
This monograph is motivated by the lack of a commercial agile and adaptable object
detection and tracking framework for UAVs equipped with a visual sensor. Existing
solutions are often application specific and implemented ad hoc, resulting in
rigid object detection and tracking systems which are difficult to adapt and reconfigure
for use in other similar applications. This thesis presents a novel framework
for object detection and tracking in UAVs equipped with a visual sensor where the
focus is to make it as configurable, modularized and as hardware independent as
possible.
Chapter 2 presents a modularized object detection and tracking framework consisting
of a path planner, machine vision and object handler modules. Each module
solves a specific sub-problem of the overall object detection and tracking problem,
and each module has a wide range of applicable solutions which could be implemented
and mixed with the remainder of the system. Chapter 2 further describes
a software toolchain which provides a bridge and abstraction layer for the object
detection and tracking framework’s architecture and the UAV’s on-board hardware components. This makes the software toolchain compatible with the idea of easily
being able to configure the individual modules and hardware of the overall UAV
object detection and tracking system. This is exemplified by the implementation of
the hardware and software modules of the object detection and tracking framework
in two different UAV platforms.
In Chapter 3, a path planner module based on model predictive control (MPC)
is developed. This controller uses a kinematic model of the UAV with certain
constraints on the UAV’s motion to generate a feasible object tracking path for the
UAV. That is, the MPC assumes that the visual sensor is mounted in a gimbal with
two degrees of freedom and supplies the on-board autopilot with waypoints and
gimbal control input which enables the UAV to track an object or group of objects.
The gimbal is controlled in a way to ensure that the gimbal is pointing towards
the tracked object’s estimated position. The MPC is demonstrated through field
experiments in Chapter 3 and 5 to be able to successfully make both of the UAV
platforms developed in this thesis able to track objects.
Chapter 4 develops a solution for the machine vision module as an object detection,
recognition and tracking module compatible with the developed object detection
and tracking framework. The module enables the UAV to perform on-board image
processing of thermal images in order to automatically segment the images
into either background or objects of interest. Using the on-board telemetry data
indicating the UAVs position and attitude coupled with the segmented images, the
module is able to use a Kalman filter to track and estimate the position and velocity
of the detected objects of interest.
Chapter 5 demonstrates that the developed object detection and tracking framework
is agile and adaptable by replacing the machine vision module of the system
with a module more suitable for use in a sea ice management application. Furthermore,
the nominal tests of the alternative modules developed in this chapter show
promising results for the use of UAVs in the area of sea ice management.
A full scale test of the object detection and tracking framework where all of the
modules of the framework are activated and interacting simultaneously is still a test
that has to be conducted, although the nominal testing in Chapter 5 demonstrates
the viability of the framework and its implemented modules. | nb_NO |
