2021 journal article
Reduced-dimensional reinforcement learning control using singular perturbation approximations
AUTOMATICA, 126.
author keywords: Reinforcement learning; Linear quadratic regulator; Singular perturbation; Model-free control; Model reduction
TL;DR:
A model-free, reduced-dimensional reinforcement learning (RL) based optimal control designs for linear time-invariant singularly perturbed (SP) systems and develops both centralized and cluster-wise block-decentralized RL controllers for such networks, in reduced dimensions.
(via Semantic Scholar)
arxiv.org (repository)
Source: Web Of Science
Added: April 5, 2021
We present a set of model-free, reduced-dimensional reinforcement learning (RL) based optimal control designs for linear time-invariant singularly perturbed (SP) systems. We first present a state-feedback and output-feedback based RL control design for a generic SP system with unknown state and input matrices. We take advantage of the underlying time-scale separation property of the plant to learn a linear quadratic regulator (LQR) for only its slow dynamics, thereby saving a significant amount of learning time compared to the conventional full-dimensional RL controller. We analyze the sub-optimality of the design using SP approximation theorems and provide sufficient conditions for closed-loop stability. Thereafter, we extend both designs to clustered multi-agent consensus networks, where the SP property reflects through clustering. We develop both centralized and cluster-wise block-decentralized RL controllers for such networks, in reduced dimensions. We demonstrate the details of the implementation of these controllers using simulations of relevant numerical examples and compare them with conventional RL designs to show the computational benefits of our approach.
