Total least squares from a Bayesian perspective: Incorporating data-informed forgetting

Abstract

The real-time estimation of error-in-variables (EIV) models with unknown time-varying parameters is considered and resolved using a Bayesian framework. The stochastic model under consideration is a regression-type model that accounts for inherently inaccurate measurements, which are corrupted by the normal noise. The EIV model identification is traditionally performed via total least squares (TLS), relying on computationally intensive methods to numerically obtain a point estimate. Such a concept, despite its theoretical appeal, nevertheless lacks the ability to quantify the uncertainty associated with the parameter estimates. Thus, this limitation hinders the concept from being combined with the statistical decision-making strategies. The paper opens the way towards enriching the standard TLS in this respect. The enrichment is achieved by projecting the unnormalized posterior generated by the EIV parametric models onto the normal-Wishart distribution. This projection is made optimal by minimizing the Kullback-Leibler distance between the unnormalized and the normal-Wishart posteriors while imposing a hard equality constraint on the mean parameter scalar product. By establishing credible intervals for both the regression parameters and the noise precision, the resultant procedure is additionally endowed with Bayesian data-informed forgetting, which allows for effective operation in nonstationary environments.