Vision-Based Control of a Robot Interacting with Moving and Flexible Objects
Abstract
Industrial tasks that involves manipulation of moving or flexible objects are often performed
with manual labour, while the use of industrial robots have mostly been limited to manipulation
of static objects and objects moving at constant velocity. The goal of this thesis is to present new
methods and solutions that will extend the capabilities of industrial robots to the manipulation
of moving and flexible objects. Two industrial tasks are considered. The first task is loading
of objects onto swinging trolleys suspended from an overhead conveyor. This is a common task
in industrial paint lines. The second task involves human-robot cooperative handling of long
flexible beams. This is a labour intensive task during manufacturing of extruded aluminum
products.
The solutions proposed in this thesis are based on the use of dynamic and visual models of the
objects under consideration. The configuration and velocity states are estimated with a particle
filter, and the state estimates are used to control the robot. The input to the particle filters
are image streams from cameras. In contrast to earlier work on visual tracking the dynamical
models are based on the detailed equations of motion to improve tracking performance and
robustness to visual clutter.
One of the reasons why automatic loading of objects onto trolleys has been difficult is because
the trolleys are not rigidly attached to the conveyor, which makes the trolley perform a swinging
motion. In this thesis it is proposed to use a dynamical model based on a pendulum model with
accurate parameters to predict the swinging motion of the trolley for tracking and robot control.
When objects are loaded onto the trolley the center of mass will move, which means that the
parameters of the dynamical model will be changing. A method was therefore developed to find
accurate mass center parameters during visual tracking. The objects loaded onto the trolley
will also occlude parts of the trolley, which makes it more difficult to track. A method was
therefore found to detect occlusions. The detection of occlusions is used to improve tracking by
not searching for occluded image features and can be used to detect failures during the loading
process. Extensive laboratory experiments were performed to demonstrate the performance of
the proposed methods.
Handling of long flexible beams is often performed by two persons holding the beam at each
end. In this thesis a method is proposed to replace one of the persons by a robot. The goal is
to let the robot move the beam by following the motions of the person at the other end. To
achieve this a dynamical model was used to model the flexible behaviour of the beam, using the
Absolute Nodal Coordinate formulation. A visual model was also designed, and the states of
the flexible beam were estimated using a particle filter. The results from two experiments are
presented to demonstrate the good performance of the proposed method.