3D robot vision using multiple cameras

Abstract

3D computer vision has in recent years become more popular in regards to robotic vision systems. This thesis looks into the possibility of using a multiple 3D camera setup for robotic vision. The approach is to reconstruct a scene in 3D from multiple camera viewpoints, extract an object from the scene and perform object recognition to find the position and orientation of an object with a known 3D geometry.

The output point cloud from the cameras were merged together to reconstruct the entire scene. To be able to reconstruct the scene, the extrinsic parameters for each camera was required. A system was implemented to automatically calibrate the cameras extrinsic parameters, that was further used to transform the point clouds.

A object recognition system was also implemented, based on the Point Cloud Library (PCL). This system supports the required filters and algorithm to be able to detect an object to find its position and orientation in the scene. Object recognition was achieved by implementing a proposed recognition pipeline, based on local feature descriptors. The system is also interfaced to a robot controller via Robot Operating System (ROS).

The experimental results indicate some variation in the objects position and less variation in the objects orientation. This variation appears to originate due to poor repeatability in depth measurements with the Microsoft Kinect v2. The results are believed to be accurate enough for a grasping task performed by a robot.