High-Performance Embedded Systems for Stereoscopic Vision

Abstract

Denne oppgaven har som mål å hente ut 3D-informasjon om omgivelsene i sanntid ved bruk av stereoskopisk datasyn. Oppgaven var utviklet for Integrated 3D Sensors (I3DS). Stereosyn består av to hovedproblemer. Det første er samsvarsproblemet (correspondence problem); for hver pixel i det ene bilde, finne dens samsvarende pixel i det andre bildet. To tilnærminger for å løse dette problemet presenteres: lokalbasert og semi-lokalbasert tilnærming. Den andre utfordringen er høy datagjennomstrømming som gjør det utfordrende å utvikle et system som gir høy bildeflyt med lavt strømtrekk. Forskjellige innvevde arkitekturer utforskes i denne oppgaven for å løse disse problemene. Bruk av maskinvareakseleratorer (FPGA og GPU) gir bedre ytelse, men samtidig vesentlig høyere utviklingskostnader og tid.

This thesis aims to extract real-time 3D information about the environment using stereoscopic vision in space; it was developed for the Integrated 3D Sensors project (I3DS). Stereo vision presents two significant challenges first is the correspondence problem; finding for every pixel it's corresponding pixel in the other image, two approaches for solving this problem are explored in this thesis local-based approach and semi-global based approach. The second challenge is the high throughput of data, making it challenging to implement a system that provides high frame rates with low power consumption, different embedded architectures are explored in this thesis; embedded CPU, embedded GPU, and FPGA to mitigate that problem. The use of hardware accelerators (FPGA and GPU) provides better performance but at a significantly higher development costs, and time, so this project exploits modern software tools to accelerate the embedded development, OpenCV on the embedded CPU, CUDA implementation of OpenCV and libsgm on the embedded GPU, and the new library by Xilinx XFopenCV synthesized for the FPGA. Finally, local block stereo matching and semi-global stereo matching are implemented on CPU, GPU and FPGA, providing six different implementations, benchmarked for stereo matching accuracy, processing time/frame rates and power consumption. It is shown that the CPU implementation provides the highest accuracy, but lowest frame rates, and highest power consumption, the FPGA implementation provided the lowest accuracy, but highest frame rates and lowest power consumption, while the GPU scored mid-way in all of its benchmarks.