Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning

Abstract: This paper introduces a method, based on deep reinforcement learning, for automatically generating a general purpose decision making function. A Deep Q-Network agent was trained in a simulated environment to handle speed and lane change decisions for a truck-trailer combination. In a highway driving case, it is shown that the method produced an agent that matched or surpassed the performance of a commonly used reference model. To demonstrate the generality of the method, the exact same algorithm was also tested by training it for an overtaking case on a road with oncoming traffic. Furthermore, a novel way of applying a convolutional neural network to high level input that represents interchangeable objects is also introduced.

Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning

The paper on "Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning" presents a method for enhancing decision-making capabilities in autonomous driving systems by employing deep reinforcement learning. The authors have applied the Deep Q-Network (DQN) algorithm to develop a versatile autonomous system capable of performing speed control and lane-change maneuvers in simulated driving environments. This approach is tested on both highway driving and overtaking scenarios, demonstrating its adaptability to different traffic conditions.

The methodological framework involves training a DQN agent which learns a policy to maximize cumulative rewards while navigating through complex driving scenarios. Reinforcement learning serves as the cornerstone for capturing optimal behaviors through a process of trial and error. The DQN agent is trained within a simulated environment to perform speed regulation and lane-changing chores for a truck-trailer combination. The paper contrasts the DQN agent’s performance with traditional rule-based models such as the Intelligent Driver Model (IDM) and the MOBIL model, showing that the agent often outperforms these baseline systems.

Technical Contributions

1. Deep Reinforcement Learning (DRL): The primary contribution is the application of DRL to develop a decision-making function for vehicle automation. By training in a simulated environment, the DQN algorithm generates high-quality control output that matches or exceeds the efficacy of established models.

2. General Purpose Adaptability: The method is shown to be adaptable, capable of handling different driving scenarios without the need for reconfiguration. This adaptability is demonstrated through experiments on conventional highway scenarios and overtaking cases involving oncoming traffic.

3. Innovative Neural Network Design: The authors introduce a convolutional neural network (CNN) architecture applied to high-level sensor data that represent interchangeable objects, which improves the learning and decision-making process. This contrasts with traditional applications of CNNs which focus on low-level pixel data.

Evaluation and Results

The evaluation results demonstrate that the DQN agent can achieve a performance index greater than 1, indicating superior performance compared to established approaches for lane changing and speed control. The trained agents were able to complete all simulation episodes without collisions, showing proficiency and reliability. This is a significant finding, suggesting that the method is robust provided that the training regimen encompasses diverse traffic situations.

Implications and Future Research

The implications of this research are noteworthy in the pursuit of more efficient and capable autonomous driving systems. The integration of DRL provides a pathway for systems that need to adapt to dynamic environments and varying traffic conditions without extensive human intervention in modeling behaviors. This work lays the groundwork for the application of reinforcement learning in more complex driving scenarios, such as urban driving with more intricate patterns involving intersections, pedestrian crossings, and cyclists.

Future developments might explore more advanced neural architectures and reinforcement learning strategies like prioritized experience replay to enhance performance further. Moreover, expanding testing to real-world scenarios would bridge the gap between simulated environments and practical applications. Investigating hybrid models that combine DRL with traditional rule-based systems could offer insight into balancing reactive and planned behavior.

Overall, the paper provides a strong contribution to the field of autonomous driving by harnessing machine learning advancements to develop a reactive and adaptive driving model.