Short vs. Long-term Coordination of Drones: When Distributed Optimization Meets Deep Reinforcement Learning

Abstract: Swarms of autonomous interactive drones can provide compelling sensing capabilities in Smart City applications, such as traffic monitoring. This paper focuses on the task assignment problem for large-scale spatio-temporal sensing by a drone swarm. However, existing approaches have distinct challenges: distributed evolutionary optimization, such as collective learning, lacks long-term adaptability in dynamic environments, while deep reinforcement learning (DRL) is limited to scale effectively due to the curse of dimensionality. Therefore, this paper proposes a novel synergetic optimization approach by integrating long-term DRL and short-term collective learning. Through this approach, each drone independently and proactively determines its flying direction and recharging location using DRL, while evolving their navigation and sensing policies through collective learning based on a structured tree communication model. Extensive experiments with datasets generated from realistic urban mobility demonstrate an outstanding performance of the proposed solution in complex scenarios. New insights show that this approach provides a win-win synthesis of short-term and long-term strategies for drone-based traffic monitoring, with short-term methods addressing training complexity and energy management, while long-term methods preserving high sensing performance.