Bilevel programming as a means of infinite weighting in regression problems

Abstract

Linear regression is concerned about fitting a model to a set of data. The weighted least squares method is a standard tool for performing linear regression. In this paper, we focus on the case when some of the samples are given priority over others. The residuals for these samples should be given an infinite weighting compared to other samples. However, due to numerical limitations, a weight which is finite but sufficiently large must be chosen instead. We suggest an alternative approach that in practice allows infinite weighting. This is achieved by reformulating the regression optimization problem as a bilevel program. The method is illustrated in a numerical example study. The example shows that, without needing to determine a weighting factor, the proposed method yields the same solution, up to numerical precision, as to the one obtained by using a large weight.