2022 article
Multi-Stage Hybrid Federated Learning Over Large-Scale D2D-Enabled Fog Networks
Hosseinalipour, S., Azam, S. S., Brinton, C. G., Michelusi, N., Aggarwal, V., Love, D. J., & Dai, H. (2022, February 3). IEEE-ACM TRANSACTIONS ON NETWORKING.
author keywords: Collaborative work; Device-to-device communication; Training; Servers; Topology; Computational modeling; Convergence; Fog learning; device-to-device communications; peer-to-peer learning; cooperative learning; distributed machine learning; semi-decentralized federated learning
TL;DR:
This work develops multi-stage hybrid federated learning (<monospace>MH-FL</monospace), a hybrid of intra-and inter-layer model learning that considers the network as a multi-layer cluster-based structure and derives the upper bound of convergence for MH-FL with respect to parameters of the network topology.
(via Semantic Scholar)
arxiv.org (repository)
UN Sustainable Development Goal Categories
11. Sustainable Cities and Communities
(Web of Science)
Source: Web Of Science
Added: March 7, 2022
Federated learning has generated significant interest, with nearly all works focused on a “star” topology where nodes/devices are each connected to a central server. We migrate away from this architecture and extend it through the <italic>network</italic> dimension to the case where there are multiple layers of nodes between the end devices and the server. Specifically, we develop multi-stage hybrid federated learning (<monospace>MH-FL</monospace>), a hybrid of intra-and inter-layer model learning that considers the network as a <italic>multi-layer cluster-based structure.</italic> <monospace>MH-FL</monospace> considers the <italic>topology structures</italic> among the nodes in the clusters, including local networks formed via device-to-device (D2D) communications, and presumes a <italic>semi-decentralized architecture</italic> for federated learning. It orchestrates the devices at different network layers in a collaborative/cooperative manner (i.e., using D2D interactions) to form <italic>local consensus</italic> on the model parameters and combines it with multi-stage parameter relaying between layers of the tree-shaped hierarchy. We derive the upper bound of convergence for <monospace>MH-FL</monospace> with respect to parameters of the network topology (e.g., the spectral radius) and the learning algorithm (e.g., the number of D2D rounds in different clusters). We obtain a set of policies for the D2D rounds at different clusters to guarantee either a finite optimality gap or convergence to the global optimum. We then develop a distributed control algorithm for <monospace>MH-FL</monospace> to tune the D2D rounds in each cluster over time to meet specific convergence criteria. Our experiments on real-world datasets verify our analytical results and demonstrate the advantages of <monospace>MH-FL</monospace> in terms of resource utilization metrics.