A new pipeline uses large language models to convert real crash reports into test scenarios for autonomous driving systems.
Researchers fed historical failure records from the NHTSA ADS crash database into a modular LLM-based system that extracts categorical and contextual information from natural language reports. The pipeline then generates synthetic scenarios compatible with a given system's testing constraints. Applied to the Metadrive simulator, it produced scenarios spanning 4 road types and 3 non-ego vehicle movement types, including work-zone anomalies. The team found meaningful system failures using just 20 generated scenarios.
Current simulation-based AV testing typically relies on mathematical search over fixed scenario representations — an approach that can miss the messy, specific conditions that actually cause crashes. By anchoring scenario generation to documented real-world failures, this method sidesteps the blank-canvas problem and targets conditions that have already proven dangerous. That's a meaningful shift in how the industry might use public crash data.
Automakers and regulators have long collected ADS incident reports without a clear automated path from record to test case — this work suggests that path now exists, though scaling beyond 20 scenarios and validating transfer to real vehicles remain open questions.