Deep Learning based frameworks for 3D registration of differential and multimodal data

Abstract

In recent decades, Visual Computing methodologies such as image processing and computer vision have addressed problems in the field of Cultural Heritage (CH)resulting in significant benefits. Specifically, accurate scanning methods have proved invaluable for documenting cultural heritage assets. However, such scans can also be used to track changes over time and to create holistic models of CH artefacts, resulting from multiple scan modalities. This in turn necessitates solving specific challenges in the task of registration, a classic problem in Visual Computing.

Informally, registration is the action of placing two geometric datasets with overlap(e.g. point clouds) in a common reference frame so that the areas of overlap match as closely as possible. This thesis focuses on two special cases of 3D registration: cross-time and multimodal. The first research area concerns the registration of differential 3D data, where the object of interest may have changed over time. The second research area concerns the registration of data from different modalities; specifically 3D point clouds and micro-CT volumes have been addressed. As both problems are too complex to address with direct algorithms while training instances exist or can be generated, it was chosen to apply deep learning methodologies to solve them and the results have been very encouraging.

Additionally, the cross-time registration solution has been extended into an auto-mated framework for change monitoring and difference detection for CH objects, while the multimodal method was combined with the cross-time method in order to monitor changes on both the surface and inner structure of CH objects.