Papers
Topics
Authors
Recent
Search
2000 character limit reached

Near-optimal Differentially Private Client Selection in Federated Settings

Published 13 Oct 2023 in cs.CR, cs.AI, and cs.DC | (2310.09370v1)

Abstract: We develop an iterative differentially private algorithm for client selection in federated settings. We consider a federated network wherein clients coordinate with a central server to complete a task; however, the clients decide whether to participate or not at a time step based on their preferences -- local computation and probabilistic intent. The algorithm does not require client-to-client information exchange. The developed algorithm provides near-optimal values to the clients over long-term average participation with a certain differential privacy guarantee. Finally, we present the experimental results to check the algorithm's efficacy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proceedings of International Conference on Artificial Intelligence and Statistics, vol. 54, 2017, pp. 1273–1282.
  2. J. Konecny, B. McMahan, and D. Ramage, “Federated optimization: Distributed optimization beyond the datacenter,” in NIPS Workshop on Optimization for Machine Learning, 2015.
  3. P. Kairouz, H. B. McMahan, B. Avent, and et al., “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021.
  4. C. Dwork, “Differential privacy,” in Automata, Languages and Programming, M. Bugliesi, B. Preneel, V. Sassone, and I. Wegener, Eds., 2006, pp. 1–12.
  5. C. Dwork and A. Roth, “The algorithmic foundations of differential privacy,” Found. Trends Theor. Comput. Sci., vol. 9, no. 3–4, pp. 211–407, 2014.
  6. L. Sankar, S. R. Rajagopalan, S. Mohajer, and H. V. Poor, “Smart meter privacy: A theoretical framework,” IEEE Transactions on Smart Grid, vol. 4, no. 2, pp. 837–846, 2013.
  7. G. A. Kaissis, M. R. Makowski, D. Ruckert, and R. F. Braren, “Secure, privacy-preserving and federated machine learning in medical imaging,” Nature Machine Intelligence, vol. 2, pp. 305–311, 2020.
  8. U. Erlingsson, V. Pihur, and A. Korolova, “Rappor: Randomized aggregatable privacy-preserving ordinal response,” in Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, 2014, pp. 1054–1067.
  9. B. Ding, J. Kulkarni, and S. Yekhanin, “Collecting telemetry data privately,” in Advances in Neural Information Processing Systems, 2017.
  10. A. G. Thakurta, A. H. Vyrros, U. Vaishampayan, G. Kapoor, J. Freudiger, V. R. Sridhar, and D. Davidson, “Learning new words,” March 2017, US Patent 9,594,741.
  11. W. M. Griggs, J. Y. Yu, F. R. Wirth, F. Hausler, and R. Shorten, “On the design of campus parking systems with QoS guarantees,” IEEE Trans. Intelligent Transportation Systems, vol. 17, no. 5, pp. 1428–1437, 2016.
  12. S. L. Warner, “Randomized response: A survey technique for eliminating evasive answer bias,” Journal of the American Statistical Association, vol. 60, no. 309, pp. 63–69, 1965.
  13. J. C. Duchi, M. I. Jordan, and M. J. Wainwright, “Local privacy and statistical minimax rates,” in IEEE Annual Symposium on Foundations of Computer Science, 2013, pp. 429–438.
  14. P. Kairouz, S. Oh, and P. Viswanath, “Extremal mechanisms for local differential privacy,” J. Mach. Learn. Res., vol. 17, no. 1, pp. 492–542, 2016.
  15. Y. Wang, X. Wu, and D. Hu, “Using randomized response for differential privacy preserving data collection,” in Proceedings of the Workshops of the EDBT/ICDT Joint Conference, vol. 1558, 2016.
  16. T. Ding, S. Zhu, J. He, C. Chen, and X. Guan, “Consensus-based distributed optimization in multi-agent systems: Convergence and differential privacy,” in IEEE Conference on Decision and Control (CDC), 2018, pp. 3409–3414.
  17. C. Zhang, M. Ahmad, and Y. Wang, “Privacy-preserving decentralized optimization based on admm,” IEEE Transactions on Information Forensics and Security, vol. 14, no. 3, 2019.
  18. R. Dobbe, Y. Pu, J. Zhu, K. Ramchandran, and C. Tomlin, “Local differential privacy for multi-agent distributed optimal power flow,” in IEEE PES Innovative Smart Grid Technologies Europe (ISGT-Europe), 2020, pp. 265–269.
  19. S. Han, U. Topcu, and G. J. Pappas, “Differentially private distributed constrained optimization,” IEEE Transactions on Automatic Control, vol. 62, no. 1, pp. 50–64, 2017.
  20. Z. Huang, S. Mitra, and N. Vaidya, “Differentially private distributed optimization,” in Proceedings of the International Conference on Distributed Computing and Networking, ser. ICDCN, no. 4, 2015.
  21. O. Beaude, P. Benchimol, S. Gaubert, P. Jacquot, and N. Oudjane, “A privacy-preserving method to optimize distributed resource allocation,” SIAM Journal on Optimization, vol. 30, no. 3, pp. 2303–2336, 2020.
  22. J. Le Ny and G. J. Pappas, “Differentially private filtering,” IEEE Transactions on Automatic Control, vol. 59, no. 2, pp. 341–354, 2014.
  23. V. Katewa, F. Pasqualetti, and V. Gupta, “On privacy vs. cooperation in multi-agent systems,” International Journal of Control, vol. 91, no. 7, pp. 1693–1707, 2018.
  24. F. Fioretto, L. Mitridati, and P. V. Hentenryck, “Differential privacy for stackelberg games,” in Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, 2020, pp. 3480–3486.
  25. L. Chen, B. Ghazi, R. Kumar, and P. Manurangsi, “On distributed differential privacy and counting distinct elements,” in Innovations in Theoretical Computer Science (ITCS), 2021.
  26. S. E. Alam, D. Shukla, and S. Rao, “A communication-efficient local differentially private algorithm in federated optimization,” IEEE Access, vol. 11, pp. 58 254–58 268, 2023.
  27. S. E. Alam and D. Shukla, “Communication-efficient allocation of multiple indivisible resources in a federated multi-agent system,” in IEEE Conference on Decision and Control, Singapore, December 2023.
  28. S. E. Alam, “Communication-efficient distributed multi-resource allocation,” Ph.D. dissertation, Concordia University, Montreal, Quebec, Canada, 2022. [Online]. Available: https://spectrum.library.concordia.ca/id/eprint/989944/
  29. S. E. Alam, R. Shorten, F. Wirth, and J. Y. Yu, “On the control of agents coupled through shared unit-demand resources,” arXiv:1803.10386 [cs.SY], 2019.
  30. S. Q. Zhang, J. Lin, and Q. Zhang, “A multi-agent reinforcement learning approach for efficient client selection in federated learning,” in Thirty-Sixth Conference on Artificial Intelligence, AAAI, 2022, pp. 9091–9099.
  31. S. E. Alam and D. Shukla, “Optimal regulation of prosumers and consumers in smart energy communities,” in IEEE International Smart Cities Conference, ISC2, Pafos, Cyprus, 2022, pp. 1–7.
  32. G. D. Németh, M. A. Lozano, N. Quadrianto, and N. M. Oliver, “A snapshot of the frontiers of client selection in federated learning,” Transactions on Machine Learning Research, 2022.
  33. S. P. Kasiviswanathan, H. K. Lee, K. Nissim, S. Raskhodnikova, and A. Smith, “What can we learn privately?” in Annual IEEE Symposium on Foundations of Computer Science, 2008, pp. 531–540.
Citations (3)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up