Distributed Autonomous Swarm Formation for Dynamic Network Bridging
Abstract: Effective operation and seamless cooperation of robotic systems are a fundamental component of next-generation technologies and applications. In contexts such as disaster response, swarm operations require coordinated behavior and mobility control to be handled in a distributed manner, with the quality of the agents' actions heavily relying on the communication between them and the underlying network. In this paper, we formulate the problem of dynamic network bridging in a novel Decentralized Partially Observable Markov Decision Process (Dec-POMDP), where a swarm of agents cooperates to form a link between two distant moving targets. Furthermore, we propose a Multi-Agent Reinforcement Learning (MARL) approach for the problem based on Graph Convolutional Reinforcement Learning (DGN) which naturally applies to the networked, distributed nature of the task. The proposed method is evaluated in a simulated environment and compared to a centralized heuristic baseline showing promising results. Moreover, a further step in the direction of sim-to-real transfer is presented, by additionally evaluating the proposed approach in a near Live Virtual Constructive (LVC) UAV framework.
- J. Jiang, C. Dun, T. Huang, and Z. Lu, “Graph convolutional reinforcement learning,” in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=HkxdQkSYDB
- P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in International Conference on Learning Representations, 2018. [Online]. Available: https://openreview.net/forum?id=rJXMpikCZ
- M. J. Hausknecht and P. Stone, “Deep recurrent q-learning for partially observable mdps,” in 2015 AAAI Fall Symposia, Arlington, Virginia, USA, November 12-14, 2015. AAAI Press, 2015, pp. 29–37.
- Y. Zhou, N. Cheng, N. Lu, and X. S. Shen, “Multi-uav-aided networks: Aerial-ground cooperative vehicular networking architecture,” IEEE Vehicular Technology Magazine, vol. 10, pp. 36–44, 12 2015. [Online]. Available: http://ieeexplore.ieee.org/document/7317860/
- M. Tortonesi, C. Stefanelli, E. Benvegnu, K. Ford, N. Suri, and M. Linderman, “Multiple-uav coordination and communications in tactical edge networks,” IEEE Communications Magazine, vol. 50, pp. 48–55, 10 2012.
- S. J. Park, H. Kim, K. Kim, and H. Kim, “Drone formation algorithm on 3d space for a drone-based network infrastructure,” 2016 IEEE 27th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), pp. 1–6, 2016.
- S. E. Hammami, H. Afifi, H. Moungla, and A. Kamel, “Drone-assisted cellular networks: A multi-agent reinforcement learning approach,” IEEE International Conference on Communications, vol. 2019-May, 5 2019.
- B. Jiang, S. N. Givigi, and J. A. Delamer, “A marl approach for optimizing positions of vanet aerial base-stations on a sparse highway,” IEEE Access, vol. 9, pp. 133 989–134 004, 2021.
- F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE Transactions on Neural Networks, vol. 20, no. 1, pp. 61–80, 2009.
- Z. Wang, T. Schaul, M. Hessel, H. Van Hasselt, M. Lanctot, and N. De Freitas, “Dueling network architectures for deep reinforcement learning,” in Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ser. ICML’16. JMLR.org, 2016, p. 1995–2003.
- R. Galliera, M. Zaccarini, A. Morelli, R. Fronteddu, F. Poltronieri, N. Suri, and M. Tortonesi, “Learning to sail dynamic networks: The marlin reinforcement learning framework for congestion control in tactical environments,” in MILCOM 2023 - 2023 IEEE Military Communications Conference (MILCOM), 2023, pp. 424–429.
- A. Amato, R. Fronteddu, and N. Suri, “Dynamically creating tactical network emulation scenarios using unity and emane,” in MILCOM 2023 - 2023 IEEE Military Communications Conference (MILCOM), 2023, pp. 201–206.
- A. Chen, K. Mitsopoulos, and R. Romagnoli, “Reinforcement learning-based optimal control and software rejuvenation for safe and efficient uav navigation,” in 2023 62nd IEEE Conference on Decision and Control (CDC), 2023, pp. 7527–7532.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.