Papers
Topics
Authors
Recent
Search
2000 character limit reached

Formal Logic Enabled Personalized Federated Learning Through Property Inference

Published 15 Jan 2024 in cs.AI and cs.LG | (2401.07448v2)

Abstract: Recent advancements in federated learning (FL) have greatly facilitated the development of decentralized collaborative applications, particularly in the domain of Artificial Intelligence of Things (AIoT). However, a critical aspect missing from the current research landscape is the ability to enable data-driven client models with symbolic reasoning capabilities. Specifically, the inherent heterogeneity of participating client devices poses a significant challenge, as each client exhibits unique logic reasoning properties. Failing to consider these device-specific specifications can result in critical properties being missed in the client predictions, leading to suboptimal performance. In this work, we propose a new training paradigm that leverages temporal logic reasoning to address this issue. Our approach involves enhancing the training process by incorporating mechanically generated logic expressions for each FL client. Additionally, we introduce the concept of aggregation clusters and develop a partitioning algorithm to effectively group clients based on the alignment of their temporal reasoning properties. We evaluate the proposed method on two tasks: a real-world traffic volume prediction task consisting of sensory data from fifteen states and a smart city multi-task prediction utilizing synthetic data. The evaluation results exhibit clear improvements, with performance accuracy improved by up to 54% across all sequential prediction models.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. Guiding Federated Learning with Inferenced Formal Logic Properties. In Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023), ICCPS ’23, 274–275. New York, NY, USA: Association for Computing Machinery. ISBN 9798400700361.
  2. Federated learning with personalization layers. arXiv preprint arXiv:1912.00818.
  3. Survey on mining signal temporal logic specifications. Information and Computation, 104957.
  4. pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learning. arXiv preprint arXiv:2206.03655.
  5. Exploiting shared representations for personalized federated learning. In International Conference on Machine Learning, 2089–2099. PMLR.
  6. Adaptive personalized federated learning. arXiv preprint arXiv:2003.13461.
  7. Integrating prior knowledge into deep learning. In 2017 16th IEEE international conference on machine learning and applications (ICMLA), 920–923. IEEE.
  8. Robust satisfaction of temporal logic over real-valued signals. In Formal Modeling and Analysis of Timed Systems: 8th International Conference, FORMATS 2010, Klosterneuburg, Austria, September 8-10, 2010. Proceedings 8, 92–106. Springer.
  9. Personalized federated learning: A meta-learning approach. arXiv preprint arXiv:2002.07948.
  10. FHWA. 2016 [Online]. Highway Performance Monitoring System Field Manual. Office of Highway Policy Information.
  11. An efficient framework for clustered federated learning. Advances in Neural Information Processing Systems, 33: 19586–19597.
  12. Distilling the Knowledge in a Neural Network. arXiv:1503.02531.
  13. Harnessing Deep Neural Networks with Logic Rules. arXiv:1603.06318.
  14. Telex: Passive stl learning using only positive examples. In International Conference on Runtime Verification, 208–224. Springer.
  15. Physics-guided machine learning for scientific discovery: An application in simulating lake temperature profiles. ACM/IMS Transactions on Data Science, 2(3): 1–26.
  16. SUMO (Simulation of Urban MObility)-an open-source traffic simulation. In Proceedings of the 4th middle East Symposium on Simulation and Modelling (MESM20002), 183–187.
  17. Ditto: Fair and robust federated learning through personalization. In International Conference on Machine Learning, 6357–6368. PMLR.
  18. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3): 50–60.
  19. Fedbn: Federated learning on non-iid features via local batch normalization. arXiv preprint arXiv:2102.07623.
  20. A novel spatial–temporal specification-based monitoring system for smart cities. IEEE Internet of Things Journal, 8(15): 11793–11806.
  21. STLnet: Signal temporal logic enforced multivariate recurrent neural networks. Advances in Neural Information Processing Systems, 33: 14604–14614.
  22. Data sets, modeling, and decision making in smart cities: A survey. ACM Transactions on Cyber-Physical Systems, 4(2): 1–28.
  23. Cityguard: A watchdog for safety-aware conflict detection in smart cities. In Proceedings of the Second International Conference on Internet-of-Things Design and Implementation, 259–270.
  24. Predictive monitoring with logic-calibrated uncertainty for cyber-physical systems. ACM Transactions on Embedded Computing Systems (TECS), 20(5s): 1–25.
  25. Cityresolver: a decision support system for conflict resolution in smart cities. In 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS), 55–64. IEEE.
  26. Toward formal methods for smart cities. Computer, 54(9): 39–48.
  27. Monitoring temporal properties of continuous signals. In Formal Techniques, Modelling and Analysis of Timed and Fault-Tolerant Systems, 152–166. Springer.
  28. Three approaches for personalization with applications to federated learning. arXiv preprint arXiv:2002.10619.
  29. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, 1273–1282. PMLR.
  30. Agnostic federated learning. In International Conference on Machine Learning, 4615–4625. PMLR.
  31. Incorporating prior domain knowledge into deep neural networks. In 2018 IEEE international conference on big data (big data), 36–45. IEEE.
  32. Federated learning for internet of things: A comprehensive survey. IEEE Communications Surveys & Tutorials, 23(3): 1622–1658.
  33. A review of cognitive assistants for healthcare: Trends, prospects, and future directions. ACM Computing Surveys (CSUR), 53(6): 1–37.
  34. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems.
  35. Informed Machine Learning–A taxonomy and survey of integrating prior knowledge into learning systems. IEEE Transactions on Knowledge and Data Engineering, 35(1): 614–633.
  36. Salvaging federated learning by local adaptation. arXiv preprint arXiv:2002.04758.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (3)

  1. Ziyan An 
  2. Taylor T. Johnson 
  3. Meiyi Ma 

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up