Learning-based State Estimation and Control using MHE and MPC Schemes with Imperfect Models

Abstract

This paper presents a reinforcement learning-based observer/controller using Moving Horizon Estimation (MHE) and Model Predictive Control (MPC) schemes where the models used in the MHE-MPC cannot accurately capture the dynamics of the real system. We first show how an MHE cost modification can improve the performance of the MHE scheme such that a true state estimation is delivered even if the underlying MHE model is imperfect. A compatible Deterministic Policy Gradient (DPG) algorithm is then proposed to directly tune the parameters of both the estimator (MHE) and controller (MPC) in order to achieve the best closed-loop performance based on inaccurate MHE-MPC models. To demonstrate the effectiveness of the proposed learning-based estimator-controller, three numerical examples are illustrated.