CAFe: Cost and Age aware Federated Learning
Abstract: In many federated learning (FL) models, a common strategy employed to ensure the progress in the training process, is to wait for at least $M$ clients out of the total $N$ clients to send back their local gradients based on a reporting deadline $T$, once the parameter server (PS) has broadcasted the global model. If enough clients do not report back within the deadline, the particular round is considered to be a failed round and the training round is restarted from scratch. If enough clients have responded back, the round is deemed successful and the local gradients of all the clients that responded back are used to update the global model. In either case, the clients that failed to report back an update within the deadline would have wasted their computational resources. Having a tighter deadline (small $T$) and waiting for a larger number of participating clients (large $M$) leads to a large number of failed rounds and therefore greater communication cost and computation resource wastage. However, having a larger $T$ leads to longer round durations whereas smaller $M$ may lead to noisy gradients. Therefore, there is a need to optimize the parameters $M$ and $T$ such that communication cost and the resource wastage is minimized while having an acceptable convergence rate. In this regard, we show that the average age of a client at the PS appears explicitly in the theoretical convergence bound, and therefore, can be used as a metric to quantify the convergence of the global model. We provide an analytical scheme to select the parameters $M$ and $T$ in this setting.
- N. Pappas A. Kosta and V. Angelakis. 2017. Age of information: A new concept, metric, and tool. Foundations and Trends in Networking 12 (November 2017), 162–259.
- REFL: Resource-Efficient Federated Learning. In EUROSYS.
- B. Buyukates and S. Ulukus. 2021. Timely Communication in Federated Learning. In IEEE Infocom.
- Communication-Efficient Federated Learning with Adaptive Parameter Freezing. In IEEE ICDCS.
- FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction. ACM Trans. on Intell. Sys. and Tech. 13, 5 (September 2022), 1–30.
- L. Den. 2012. The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]. IEEE Signal Processing Magazine, 29, 6 (October 2012), 141–142.
- PMF: a privacy-preserving human mobility prediction framework via federated learning. ACM Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (March 2020), 10–21.
- Real-time status: How often should one update?. In IEEE Infocom.
- Federated learning: Strategies for improving communication efficiency. ACM Interact. Mob. Wearable Ubiq. Tech (2016). Available online at arXiv:1610.05492.
- Efficient Federated Learning via Guided Participant Selection. In USENIX Symposium on Operating Systems Design and Implementation.
- D. Li and X. Gong. 2023. Anarchic Federated learning with Delayed Gradient Averaging. In MOBIHOC.
- A review of applications in federated learning. Elsevier Computers & Industrial Engineering 149 (November 2020), 1–15.
- On the Convergence of FedAvg on Non-IID Data. In ICLR.
- Age-aware Communication Strategy in Federated Learning with Energy Harvesting Devices. In IEEE ICCC.
- Communication-efficient learning of deep networks from decentralized data. In AISTATS.
- P. Mitra and S. Ulukus. 2023. Timely Asynchronous Hieracrchical Federated Learning: Age of Convergence. In WiOpt.
- Age of information: A new metric for information freshness. Age of information 12 (December 2019), 1–224.
- SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning with Low Overhead. IEEE Trans. Comput, 70, 5 (May 2020), 655–668.
- Asynchronous federated optimization. (2019). Available online at arXiv:1903.03934.
- Asynchronous federated learning on heterogeneous devices: A survey. (2021). Available online at arXiv:2109:04269.
- Cost-Awaare Hierarchical Federated Learning via Over-the-Air Computing. In IEEE Globecom.
- Age-Based Scheduling Policy for Federated Learning in Mobile Edge Networks. In IEEE ICASSP.
- Age of information: An introduction and survey. IEEE Jour. Sel. Areas in Comm, 39, 5 (May 2020), 1183–1210.
- Federated Learning Hyperparameter Tuning From a System Perspective. IEEE Internet of Things Journal, 10, 16 (March 2023), 14102–14113.
- Accelerating Federated Learning on Non-IID Data Against Stragglers. In IEEE SECON.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.