A retrospective look at cross model validation and its applicability in vibrational spectroscopy

Abstract

In this paper, it is presented how Cross Model Validation (CMV), also known as double cross validation, efficiently can be applied for variable selection in spectroscopic applications. The chosen applications are FT-IR spectroscopic measurements of mixtures of marzipan and NIR spectra of diesel fuels. Standard Normal Variate (SNV) is applied as a spectral pre-treatment to reduce baseline effects in the spectra for the FT-IR data whereas 2nd derivative was applied for the diesel fuels. Variable selection based on jack-knifing and frequency of significance from Cross Model Validation is employed for identifying non-relevant spectral regions as well as providing a relevant subset for model optimization. The results show a high degree of correspondence between the objectively found wavelength bands and the reported chemical interpretation found in the literature. In addition, the stability of the models due to conservative validation with respect to predictive performance is exemplified. Finally, an example of how the use of downweighing variables ensures optimal prediction ability and detailed model interpretation is shown.