Democratizing the Federation in Federated Learning
No TL;DR found
Abstract
Federated learning (FL) is a widely acknowledged distributed training paradigm that preserves the privacy of data on participating clients, and has become the de facto standard for distributed machine learning across a large number of edge devices. Conventional FL, however, has a rather rigid design, where the server is the dominant player that selects a subset of its clients to participate in each communication round, and clients are merely followers, and are not offered the freedom to accept or decline invitations from the server to participate. In addition, clients may become unavailable or very slow due to a wide variety of reasons, yet it may take an excessive amount of time for a conventional FL server to recognize that a particular client is unavailable. In this paper, we advocate for a more pragmatic paradigm in federated learning, called democratic federated learning, to offer more freedom to both servers and clients with respect to the ability to accept or decline requests, and to explicitly request to participate. In contrast to conventional federated learning, our paradigm allows (1) both the server and clients to participate and withdraw from the federated learning process at any time; (2) the server to decide whether to reject clients' updates based on the current model convergence steps, i.e., after satisfying the minimum required clients' updates received; and (3) the clients to adjust the local epochs based on their own training and communication time. Our experimental results on a variety of datasets and models have confirmed that democratic federated learning not only accelerates the convergence process but also improves the accuracy of converged models, and serves as a foundation for future explorations into client-centric models within the FL ecosystem.