Vision-Based Control of a Robot Interacting with Moving and Flexible Objects

Abstract

Industrial tasks that involves manipulation of moving or flexible objects are often performed with manual labour, while the use of industrial robots have mostly been limited to manipulation of static objects and objects moving at constant velocity. The goal of this thesis is to present new methods and solutions that will extend the capabilities of industrial robots to the manipulation of moving and flexible objects. Two industrial tasks are considered. The first task is loading of objects onto swinging trolleys suspended from an overhead conveyor. This is a common task in industrial paint lines. The second task involves human-robot cooperative handling of long flexible beams. This is a labour intensive task during manufacturing of extruded aluminum products. The solutions proposed in this thesis are based on the use of dynamic and visual models of the objects under consideration. The configuration and velocity states are estimated with a particle filter, and the state estimates are used to control the robot. The input to the particle filters are image streams from cameras. In contrast to earlier work on visual tracking the dynamical models are based on the detailed equations of motion to improve tracking performance and robustness to visual clutter. One of the reasons why automatic loading of objects onto trolleys has been difficult is because the trolleys are not rigidly attached to the conveyor, which makes the trolley perform a swinging motion. In this thesis it is proposed to use a dynamical model based on a pendulum model with accurate parameters to predict the swinging motion of the trolley for tracking and robot control. When objects are loaded onto the trolley the center of mass will move, which means that the parameters of the dynamical model will be changing. A method was therefore developed to find accurate mass center parameters during visual tracking. The objects loaded onto the trolley will also occlude parts of the trolley, which makes it more difficult to track. A method was therefore found to detect occlusions. The detection of occlusions is used to improve tracking by not searching for occluded image features and can be used to detect failures during the loading process. Extensive laboratory experiments were performed to demonstrate the performance of the proposed methods. Handling of long flexible beams is often performed by two persons holding the beam at each end. In this thesis a method is proposed to replace one of the persons by a robot. The goal is to let the robot move the beam by following the motions of the person at the other end. To achieve this a dynamical model was used to model the flexible behaviour of the beam, using the Absolute Nodal Coordinate formulation. A visual model was also designed, and the states of the flexible beam were estimated using a particle filter. The results from two experiments are presented to demonstrate the good performance of the proposed method.