A Dual Control Approach to Solve Exploration Versus Exploitation Trade-Offs in the Design of Personalized Physical Exercise Sessions

Abstract

Using biofeedback in medical therapies has proven to be effective for adapting patient behaviors while keeping the patients engaged and motivated in an exercise session. This letter considers general problems in personalized exercise sessions where the input is opportune biofeedback and the session goal is to maximize a particular exercise effect. Due to the individual differences between patients and their physiological signals, however, personalized patient models also need to be identified. With the two objectives: 1) maximize a training effect with minimal control effort, and 2) identify the individualized patient model, we have a typical exploration vs. exploration trade-off. Control problems of this form are called dual control problems. In this letter, we formulate a dual control problem for a personalized exercise session and test the approach against classical optimal control and optimal experimental design approaches in an illustrative example of performing Kegel exercises where the control and identification goals conflict with each other.