Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver Behavior for Effective On-Ramp Merging

Abstract: Highway merging scenarios featuring mixed traffic conditions pose significant modeling and control challenges for connected and automated vehicles (CAVs) interacting with incoming on-ramp human-driven vehicles (HDVs). In this paper, we present an approach to learn an approximate information state model of CAV-HDV interactions for a CAV to maneuver safely during highway merging. In our approach, the CAV learns the behavior of an incoming HDV using approximate information states before generating a control strategy to facilitate merging. First, we validate the efficacy of this framework on real-world data by using it to predict the behavior of an HDV in mixed traffic situations extracted from the Next-Generation Simulation repository. Then, we generate simulation data for HDV-CAV interactions in a highway merging scenario using a standard inverse reinforcement learning approach. Without assuming a prior knowledge of the generating model, we show that our approximate information state model learns to predict the future trajectory of the HDV using only observations. Subsequently, we generate safe control policies for a CAV while merging with HDVs, demonstrating a spectrum of driving behaviors, from aggressive to conservative. We demonstrate the effectiveness of the proposed approach by performing numerical simulations.

• The paper introduces an encoder-decoder network that forms an approximate information state to predict HDV trajectories.
• It integrates model predictive control with learned behavior predictions to optimize safe and efficient on-ramp merging.
• Validation using both real NGSIM data and simulations demonstrated the robustness and adaptability of the proposed framework.

Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver Behavior for Effective On-Ramp Merging

The paper "Connected and Automated Vehicles in Mixed-Traffic: Learning Human Driver Behavior for Effective On-Ramp Merging" (2304.00397) presents a comprehensive study focused on integrating connected and automated vehicles (CAVs) into environments dominated by human-driven vehicles (HDVs). The research investigates methods to enhance highway safety and efficiency, particularly in on-ramp merging scenarios, through dynamics modeling and control strategies.

Introduction and Objectives

The adoption of CAV technology promises significant gains in traffic safety, energy efficiency, and commuter comfort. The critical challenge addressed in this paper is the interaction between CAVs and HDVs on highways, where human drivers' unpredictability poses substantial control difficulties. Existing literature suggests various modeling techniques, including game-theoretic models and machine learning approaches such as LSTMs, to predict HDV behaviors. This paper introduces an innovative approach leveraging approximate information states, which enhances the CAV's ability to predict HDV trajectories and apply optimal control policies even in the absence of complete knowledge of HDV dynamics.

Methodology

The research proposes a multi-step process to address the on-ramp merging challenge:

1. Behavioral Learning and Prediction: The CAV uses an encoder-decoder neural network architecture to learn HDV behaviors in merging scenarios. This structure is designed to form an approximate information state that serves as a basis for predicting HDV trajectories over given horizons without prior knowledge of HDV dynamics.
2. Model Predictive Control (MPC): Utilizing the predictions from the learned models, an MPC framework is implemented. This approach enables the CAV to compute control strategies that optimize safety and energy efficiency, considering potential interaction dynamics with HDVs.
3. Validation Through Real and Simulated Data: The approach is validated using real-world data from the NGSIM repository and simulation data generated via inverse reinforcement learning. This dual validation ensures robustness across a variety of driving conditions.

   Figure 1: A visualization of the encoder-decoder architecture.

Results and Discussion

The proposed approach demonstrated significant efficacy in predicting HDV trajectories and generating safe CAV control strategies. Key findings include:

• Trajectory Predictions: The model successfully predicted HDV positions with substantial accuracy, which is critical for maintaining safe distances and reducing potential conflicts at merging points.
• Control Strategy Effectiveness: CAVs employing the model-derived strategies were able to navigate complex merging scenarios safely, exhibiting driving behaviors ranging from aggressive to conservative as necessitated by context.

The robustness of the model was particularly evident in simulations using a range of HDV driving styles, underscoring its adaptability and potential for widespread application in mixed-traffic environments.

Figure 2: The results of trajectory prediction v/s are the actual trajectory from the generated dataset.

Implications and Future Work

This research contributes substantively to the field of intelligent vehicle control within mixed-traffic systems. The practical implications are vast, suggesting pathways to safer, more efficient highway operations during critical merging activities. Theoretically, it advances the application of reinforcement learning and POMDP frameworks in vehicle control.

Future research directions include extending this approach to more complex traffic scenarios such as multi-lane highways and intersections, and testing the framework's efficacy in real-world environments beyond simulation.

Conclusion

The paper makes a significant contribution to the integration of CAVs in human-dominated road environments by combining machine learning-based prediction with model predictive control. The dual focus on learning HDV behaviors and controlling CAV responses ensures robust performance across variable traffic conditions, setting the stage for enhanced deployment of CAVs in dynamic mixed-traffic settings.