Appearance Manipulation in Spatial Augmented Reality using Image Differences

Abstract

Rapidly emerging augmented reality technologies enable us to virtually alter appearance of objects and materials in a fast and efficient way. The state-of-the-art research shows that the human visual system has a poor ability to invert the optical processes in the scene and rather relies on images cues and spatial distribution of luminance to perceive appearance attributes, such as gloss and translucency. For this reason, we hypothesize that it is possible to alter gloss and translucency appearance by projecting an image onto the original to mimic the luminance distribution characteristic of the target appearance. To demonstrate feasibility of this approach, we use pairs of physically-based renderings of glossy and matte, and translucent and opaque materials, respectively; we calculate a compensation image – a luminance difference between them, and subsequently, we demonstrate that by algebraic addition of luminance, an image of matte object can appear glossy, and an image of opaque object can appear translucent, when respective compensation images are projected onto them. Furthermore, we introduce a novel method to increase apparent opacity of translucent materials. Finally, we propose a future direction, which could enable nearly real-time appearance manipulation.