Regularization when modeling with biased simulation data as a prior

Abstract

Embedding physical knowledge in system identification increases the generalization capabilities of the identified models. For complex engineering systems, such as a process plant, the most complete and detailed quantitative description of the existing physical and structural knowledge is often provided by a simulator. We describe the procedure of fusing simulated data with measurement data via L2 regularization for models that are linear in the parameters. We characterize how the MSE minimization problem in this framework is nontrivial, and show that for certain realizations of the data there is no unique minimum of the MSE w.r.t. the regularization parameter. In these cases the MSE can even increase to larger values than both the variance and the bias, which is counter-intuitive. We show how this issue appears less frequently with more data, even though multiple minima can occur for any realization of the data. However, we show also that the Stein effect is present regardless, so that it is always possible to decrease the MSE with careful selection of the regularization parameter, i.e., information fusion may always be beneficial.