HyperSpectral Imaging (HSI) is a vital tool to many industries and fields. It is
however very costly, time consuming, and in need of dedicated hardware. Lots of
research was dedicated to find alternatives to traditional HSI systems. One of the
most promising ones is RGB to Hyperspectral reconstruction. These models are
usually CNNs that take in a single RGB image and estimates the hyperspectral
image for the same scene (in the visible range). Such models can dramatically cut
on costs and time needed to acquire a hyperspectral image given the availability
and ease of acquiring RGB images.
However, to fully adopt such models we need to establish trust in them (or
distrust). To do that, we need to understand and explain how these models work on
a fundamental level at least. This is especially the case because these models deal
with a highly ill-posed problem of mapping only 3 RGB bands into a much larger
number of bands (typically 31) to perform this estimation. Users do not have any
evidence of how these models actually do that and how they are able to estimate
the illuminant of the scene to avoid metameric effects and how they perform the
’one-to-many’ mapping involved. In this thesis, we work on filling this major gap.
We take 7 of the most prominent RGB to hyperspectral reconstruction models and
apply many explainable AI (XAI) methods to try to understand how they work. We
classify these models based on the different ways they perform the reconstruction.
We establish points of failure where some or all models cannot perform as expected.
We establish their spatial feature area in the input image. We try to find what
kind of parameters and features they use and where in the network they use them.
We present a theory on how they do illuminant estimation and present supporting
evidences for that theory. Finally, we bring all tests together and try to break
down these models into more simple sub-models that could be replicated by simpler
explainable equivalents. We also introduce novel modifications to existing XAI
methods that allows them to be used in any hyperspectral model explainability
project in the future.
The outcomes of this work support that these models work in an intelligible
manner. Meaning that they could be understood and equated by other explainable
models. However, these models cannot be trusted all the time, since the work
shows that they fail consistently under certain conditions. This work does not fully
explain these models since some aspects are still unclear, but it does explain many
important parts and paves the way for a clearer understanding of these networks.
