Abstract

Context. Autonomous vehicles (AVs) are transforming transportation, promising increased safety and efficiency. However, their validation is complex, requiring rigorous testing methodologies. To address this complexity and ensure the safety of AVs, scenario-based testing has emerged as a crucial approach for the comprehensive validation of these systems.

Problem. The validation of AV scenarios presents a significant challenge in ensuring both comprehensiveness and realism. While current scenario validation tools provide a foundational framework, there remains considerable scope for enhancement, particularly in the domains of scenario quality assessment and the evaluation of their fidelity to real-world driving conditions. The potential discrepancies between modeled scenarios and actual vehicle trajectories may compromise the authenticity of the testing environment, potentially resulting in critical oversights in AV system evaluation.

Objective. The primary objectives of this master’s thesis are centered on the investigation of scenarios within simulations, which are critical for the development and testing of autonomous driving systems. The focus is on identifying and understanding scenarios that deviate from or are unrealistic compared to expected real-world driving behaviors. The overarching goal is to identify the specific actor parameters that contribute to the creation of these unrealistic or deviated driving situations in the simulation.

Method. This thesis presents a novel framework for analyzing and enhancing autonomous driving simulation scenarios, focusing on identifying anomalous scenarios through real-world driving data. The proposed methodology integrates an Artificial Intelligence (AI) model to automatically identify driving scenarios that deviate from real-world conditions, followed by a detailed assessment of these potentially anomalous scenarios to pinpoint specific measurements or parameters causing the lack of realism or deviation.

Result. The result involves the identification of unrealistic or deviated driving scenarios within the simulated dataset, along with determining the parameter responsible for introducing unrealism or deviation in the identified scenarios. In this thesis, 14 out of 723 simulation scenarios were identified as anomalies using a specific autoencoder model combined with a distance-based technique. Additionally, the maximum and minimum velocity parameters of the actor were assessed, which proved critical in flagging these scenarios as anomalous.

Conclusion. The methodology improves simulation dataset quality, leading to more reliable and representative simulations. This enhancement in scenario fidelity is crucial for developing robust autonomous driving systems, ensuring simulated environments accurately reflect real-world complexities. By advancing scenario quality and realism, this work contributes to more effective development processes for autonomous driving systems, ultimately fostering increased safety and reliability in autonomous vehicle technology.