On Enabling Scalable, Personalized, and Private Federated Learning

Abstract

This thesis investigates distributed machine learning for emerging Internet of Things (IoT) and Cyber-Physical Systems (CPS) applications. These applications involve large-scale data collection from distributed, often privately owned devices, which raises many logistical, moral, and legal issues if centralized processing is used. Therefore, this thesis explores how to take advantage of the local processing power of the devices and enable them to collaborate via inter-device communication to achieve a common learning goal without explicit disclosure of local data. Federated learning is a framework for distributed machine learning that can address these issues. This thesis aims to develop and analyze new federated learning algorithms that overcome some of the practical challenges of distributed machine learning encountered in IoT and CPS settings. In particular, it tackles the following challenges. • The scalability of the distributed machine learning architecture with respect to the increasing number of participating devices. • The privacy preservation of the data owners from both external and internal adversaries. • The robustness and efficiency of the distributed machine learning process under resource constraints and device failures. • The personalization of the machine learning models for different devices and tasks within the same network. The main contributions of this thesis are as follows. The first contribution of this thesis is to propose a peer-to-peer federated learning algorithm in which the par ticipating devices collaborate without the need for a coordinator while preserving their confidentiality. Secondly, we devise a robust online federated learning algorithm that can handle resource-constrained devices with sporadic participation and failures, as well as delays in a single-server architecture. Third, we develop a multi-server federated learning algorithm that supports personalized model training for device-specific tasks while preserving the data privacy of the participants. Finally, we advance peer-to-peer personalized federated learning with reinforcement learning techniques to enhance localized personalized learning.