2023 journal article
Distributed Reinforcement Learning for Networked Dynamical Systems
IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, 11(2), 1103–1115.
author keywords: Heuristic algorithms; Network systems; Convergence; Observers; Network topology; Dynamical systems; Topology; Distributed control; distributed reinforcement learning (RL); networked dynamical systems (NDS); scalability
open access via ieeexplore.ieee.org (publisher PDF)
Source: Web Of Science
Added: July 1, 2024
We propose a scalable algorithm for learning distributed optimal controllers for networked dynamical systems. Assuming that the network is comprised of nearly homogeneous subsystems, each sub-controller is trained by the local state and input information from its corresponding subsystem and filtered information from its neighbors. Thus, the costs of both learning and control become independent of the number of subsystems. We show the optimality and convergence of the algorithm for the case when the individual subsystems are identical, based on an algebraic property of such networks. Thereafter, we show the robustness of the algorithm when applied to general heterogeneous networks. The effectiveness of the design is investigated through numerical simulations.