Abstract
Lighting is fundamentally crucial for numerous useful applications. The extended reality system is capable of simulating realistic scenes with precise illumination data. However, it is difficult to consistently obtain the scene's actual lighting SPD. Instead of predicting the entire spectrum of illumination SPD, the majority of known approaches for estimating illumination focus on recovering the illumination's color using spectral images. This work explores the problem of illumination SPD estimation from sRGB images. By discarding global spatial information, we convert the problem into a vector-to-vector regression task. The deep learning model is proposed to predict the illumination SPD using only the pixel values from sRGB images. In order to alleviate the lack of training SPD data, a large sRGB image dataset along with the corresponding lighting SPD is proposed in our work. The various unique illuminations are generated by the advanced 24-channel LED lighting system, which is the first one in Europe. In order to tackle the time-consuming capturing, the virtual camera model is implemented as the data-generation tool to simulate images with a variety of lighting conditions. As a result, the proposed datasets include both real-world and synthetic data for training and evaluating the model. The results of both datasets show consistent performance across a wide range of spectra according to the different numerical evaluation metrics. In the end, the sensor-independent prediction is explored by integrating the transfer learning technique when working on data from the different cameras.
