3D Reconstruction of Gastrointestinal Regions Using Single-View Methods

Abstract

Capsule endoscopy is about to become an alternative to traditional colonoscopy. One uses a wireless camera to visualize the gastrointestinal (GI) tract. A 3D model based on image sequences obtained from wireless capsule endoscopy (WCE) can be helpful to diagnose or analyse areas of interests. We have therefore investigated the possibility to provide enhanced viewing for gastroenterologists by reconstructing 3D shapes from WCE images. The study is done on virtual graphics-based models of human GI regions. The shape from shading (SFS) method is applied to colon images and the quality of the reconstructed shapes is compared with ground truth models. WCE images suffer from uneven and dim illumination due to point light source. Therefore, we provide a method based on surface normals from reconstructed 3D models to enhance contrast particularity in images capturing larger depths by changing the illumination from point light to directional light. Images of different resolution are also tested to evaluate their effect on the quality of the 3D reconstruction. We have also tested the shape from focus (SFF) method, a possibility for future WCEs, and compared the results with SFS. Finally, enhanced images and 3D shapes recovered with both methods have been evaluated by gastroenterologists through subjective experiments. Objective experiments indicate that both methods are capable of reconstructing the 3D shapes of colon images successfully, but the SFF method is better at retaining details in the reconstructed models than the SFS method. Subjective experiments show that contrast enhanced images are highly preferred over original images. Also, having the reconstructed 3D models in addition to the images during evaluation is found to be very useful by gastroenterologists and sometimes even being preferred over the original image.