Model-based Reinforcement Learning for Variable Impedance Control

Abstract

This thesis is a collection of research work in the area of reinforcement learning and robotic manipulation. A set of new results on reinforcement learning focusing on model-based approaches and variable impedance control for compliant robotic manipulation are presented. One of the major challenges in robotic research at present is to develop real-world manipulation skills with human-level dexterity while being safe. Humans possess dexterous manipulation skills owing to the compliance properties of human motor control. Inspired by human manipulation, incorporating compliant control skills is understood as a promising way to achieve this goal. Impedance control offers an ideal framework to incorporate such compliant control skills in robotic manipulators. The variable impedance control framework is an extension of the impedance control framework where compliance can be adapted in real-time according to the task requirements. Even though there exist many approaches for impedance adaptation, designing optimal impedance adaptation laws for complex manipulation tasks is highly challenging and demands further research. Machine learning with its flexibility and scalability properties is a promising approach to designing impedance adaption law for a variable impedance controller. Reinforcement learning in particular is interesting as it enables robots to identify optimal stiffness profiles by interacting with their environment. However, reinforcement learning despite its promises has not been successful in real-world robotic applications. Low sample efficiency and a lack of safety guarantees are the most critical factors hindering its success in real-world robotic applications. Model-based reinforcement learning is a promising approach to address these deficiencies in reinforcement learning. This underlines the need for research on evaluating the applicability of model-based reinforcement learning methods and also to address the limitation of current model-based reinforcement learning approaches. This thesis is an attempt in this direction with a focus on robotic manipulation and compliant robotic control. The core research focus of this thesis is enabling compliant control in robotic manipulation leveraging modelbased reinforcement learning approaches while also addressing the deficiencies in model-based reinforcement learning. A part of the contribution of our work addresses the development of new model-based reinforcement learning methods. We explored combining conventional control approaches with reinforcement learning to develop novel model-based reinforcement learning methods. This thesis contributed to advancing the research on compliant control for robotic manipulation by developing and evaluating different model-based variable impedance learning control approaches for real-world applications. Additionally, this thesis addresses planning in robotic manipulation in the context of an uncertain environment and guarantees safety and stability in learning-based approaches relevant to robot control. The results of this have provided promising directions and insights into the area of reinforcement learning and robotic manipulation in general. This research is expected to contribute towards facilitating the use of reinforcement learning and compliant control skills in real-world robotic systems.