CoLight: Learning Network-level Cooperation for Traffic Signal Control

Abstract: Cooperation among the traffic signals enables vehicles to move through intersections more quickly. Conventional transportation approaches implement cooperation by pre-calculating the offsets between two intersections. Such pre-calculated offsets are not suitable for dynamic traffic environments. To enable cooperation of traffic signals, in this paper, we propose a model, CoLight, which uses graph attentional networks to facilitate communication. Specifically, for a target intersection in a network, CoLight can not only incorporate the temporal and spatial influences of neighboring intersections to the target intersection, but also build up index-free modeling of neighboring intersections. To the best of our knowledge, we are the first to use graph attentional networks in the setting of reinforcement learning for traffic signal control and to conduct experiments on the large-scale road network with hundreds of traffic signals. In experiments, we demonstrate that by learning the communication, the proposed model can achieve superior performance against the state-of-the-art methods.

• The paper demonstrates that integrating graph attention mechanisms with index-free model learning significantly improves urban traffic signal control, achieving travel time reductions of up to 11.69%.
• It employs multi-agent reinforcement learning to dynamically adjust signal timings based on real-time traffic data, ensuring effective cooperation among intersections.
• The method scales efficiently for city-wide deployments, validated on both synthetic data and real-world datasets from cities like New York and Hangzhou.

CoLight: Learning Network-level Cooperation for Traffic Signal Control

Introduction

Modern urban traffic management requires efficient algorithms to control traffic signals in large-scale road networks. Traditional optimization techniques often make unrealistic assumptions about traffic flow and capacity. Recent research suggests that reinforcement learning (RL), particularly multi-agent RL, can offer significant improvements by dynamically adjusting signal timings based on real-time traffic conditions. "CoLight: Learning Network-level Cooperation for Traffic Signal Control" (1905.05717) presents an innovative approach that leverages graph attention networks to address traffic signal control problems at a network scale, providing dynamic cooperation among traffic signals without relying on pre-defined offsets or assumptions.

Methodology

Graph Attention Networks

CoLight introduces graph attentional networks to facilitate communication between traffic signal controllers situated at different intersections. The attentional mechanism assigns varying importance to neighboring intersections based on current conditions, which allows the model to account for both spatial and temporal dynamics in traffic flow.

Figure 1: Illustration of index-based concatenation causing confusion in shared models.

Index-Free Model Learning

One of the principal innovations of CoLight is its index-free model learning paradigm. This approach circumvents the need for fixed indexing of neighbors, which conventional models employ, and hence avoids issues with conflicting learning. CoLight achieves this by utilizing a "mean-field" idea to average influences from neighboring intersections, thereby reducing the number of parameters and enhancing scalability.

Comprehensive Experimentation Setup

CoLight was evaluated using both synthetic and real-world datasets. Simulated networks included grids with varying traffic patterns, while real-world data encompassed intersections from cities such as New York and Hangzhou. Metrics focused on average travel time as a measure of model performance across diverse scenarios. Experiments validated the superiority of CoLight's learning approach over both RL baselines and established transportation strategies.

Results

Performance Analysis

The experiments indicated that CoLight consistently outperformed state-of-the-art methods in reducing travel times, with average improvements up to 11.69% over real-world data. The advantage was enhanced in complex urban environments with dynamic and irregular traffic patterns.

Figure 2: Framework of CoLight illustrating variation in cooperation scope and attention distribution.

Scalability

Despite the computation-intensive nature of RL, CoLight demonstrated excellent scalability. The efficient parameter sharing mechanism ensured manageable memory usage and faster convergence compared to individual RL models that scale poorly with the number of intersections. In scenarios comprising up to 200 intersections, CoLight maintained optimal performance, validating its capability for city-scale deployment.

Figure 3: Real-world road networks; displays intersections for dataset modeling.

Attention Mechanism Insights

Spatial and Temporal Distribution

An analysis of the attention scores acquired by CoLight reveals significant spatial differences among intersections. Upstream intersections often exert greater influence than downstream ones. Temporal patterns across a day exhibit changes in attention distribution, aligning well with traffic variations such as rush hours. These observations underscore the capacity of CoLight's attention mechanism to adapt dynamically to varying flow conditions.

Figure 4: Spatial attention distribution showcasing varied influence from upstream and arterial intersections.

Conclusion

CoLight significantly advances the state of traffic signal control by addressing limitations in prior approaches. Its use of graph attention mechanisms to model dynamic interactions provides a robust framework for efficient traffic management. Future enhancements could involve refining neighborhood determination and incorporating additional environmental data to further boost model accuracy.

Overall, CoLight signifies an important step towards harnessing RL for real-world urban traffic challenges, illustrating scalable and adaptable methodologies essential for modern smart cities. The research offers a promising direction for future exploration in AI-driven traffic systems.