Criticality Metrics for Automated Driving: A Review and Suitability Analysis of the State of the Art

Abstract: The large-scale deployment of automated vehicles on public roads has the potential to vastly change the transportation modalities of today's society. Although this pursuit has been initiated decades ago, there still exist open challenges in reliably ensuring that such vehicles operate safely in open contexts. While functional safety is a well-established concept, the question of measuring the behavioral safety of a vehicle remains subject to research. One way to both objectively and computationally analyze traffic conflicts is the development and utilization of so-called criticality metrics. Contemporary approaches have leveraged the potential of criticality metrics in various applications related to automated driving, e.g. for computationally assessing the dynamic risk or filtering large data sets to build scenario catalogs. As a prerequisite to systematically choose adequate criticality metrics for such applications, we extensively review the state of the art of criticality metrics, their properties, and their applications in the context of automated driving. Based on this review, we propose a suitability analysis as a methodical tool to be used by practitioners. Both the proposed method and the state of the art review can then be harnessed to select well-suited measurement tools that cover an application's requirements, as demonstrated by an exemplary execution of the analysis. Ultimately, efficient, valid, and reliable measurements of an automated vehicle's safety performance are a key requirement for demonstrating its trustworthiness.

• The paper provides a systematic review of criticality metrics, detailing their role in quantifying behavioral safety for automated driving.
• It presents a comprehensive suitability analysis that evaluates metrics based on reliability, validity, sensitivity, and specificity.
• The study demonstrates that integrating multiple metrics enhances AV safety assessments and guides future advancements in the field.

Criticality Metrics for Automated Driving: A Review and Suitability Analysis of the State of the Art

Introduction

The proliferation of automated vehicles (AVs) necessitates the establishment of robust safety assurance mechanisms, particularly behavioral safety measures critical to public trust and widespread adoption. While functional safety has been well-explored, quantifying an AV's behavioral safety in dynamic contexts remains challenging. Criticality metrics have emerged as a valuable tool in assessing traffic conflicts, pivotal for ensuring safe automated driving in mixed traffic environments. This paper reviews the state of the art regarding such metrics, providing a comprehensive analysis of their applicability and proposing methodological tools for selecting appropriate metrics tailored to specific applications.

Methodological Framework

Criticality metrics are functions designed to measure a vehicle's risk level in a given traffic situation. This paper extensively reviews the properties and applications of various criticality metrics, offering guidance for systematically selecting suitable metrics. By integrating these metrics into verification and validation frameworks, AVs can be assessed for their safety performance more effectively. The paper proposes a suitability analysis approach, empowering practitioners to identify metrics that align with their application's requirements using a methodical evaluation of the state of the art.

Key Numerical Results and Claims

The paper features an extensive evaluation framework for criticality metrics, focusing on several properties, including reliability, validity, sensitivity, and specificity. Numerical results are contingent on precise definitions and measurement scales tailored to specific contexts, ensuring metrics provide relevant safety assessments. The detailed analysis covers various metrics, from traditional time-to-collision (TTC) measures to novel methods integrating predictive models with probabilistic assessments. The proposed metrics demonstrate reliable performances under defined conditions, especially when adequately combined or adapted for specific scenario types.

Bold Claims

Despite the ambiguity surrounding the definition of criticality within the field, the paper takes a decisive stance by advocating for the adoption of systematic suitability analyses, which, according to the authors, significantly enhance metric selection processes. Additionally, it posits that integrating various metrics into a cohesive analytical framework bridges the gap between abstract safety assurance concepts and practical, quantifiable measures.

Practical and Theoretical Implications

The practical implications of this research are substantial, as it provides a structured approach enabling developers and engineers to systematically incorporate criticality metrics into their safety assurance processes. Moreover, the theoretical implications extend to enhancing the understanding of metrics' roles within automated driving paradigms, serving as foundational knowledge for future metric development.

Future Developments

Emerging trends suggest that ongoing advancements in machine learning and probabilistic models will likely transform the landscape of criticality metric applications. This evolution encourages interdisciplinary collaboration between computational model developers and transportation safety experts, aimed at refining existing metrics and exploring new methodologies for enhanced accuracy and predictive capabilities.

Conclusion

"Criticality Metrics for Automated Driving: A Review and Suitability Analysis of the State of the Art" delineates a clear pathway to a systematic integration of criticality metrics in automated driving systems. Through detailed reviews and methodological proposals, the paper provides a rich resource for future research and development, fostering the creation of safer, more efficient AV systems. Its blueprint for selecting well-suited metrics underscores the need for continued innovation and rigorous evaluation standards in the evolving field of automated vehicle safety compliance.