An autonomous driving policy that gets better the more it screws up sounds like a punchline — researchers say they have made it work.
A team publishing on arXiv proposes Rollout-Retrieval Lifelong Policy Learning, or R2LPL, a framework that watches a deployed driving policy make recoverable mistakes, extracts what a correct response would have looked like, and feeds that back as supervised training data. Most learning-based autonomous driving systems are trained on expert demonstrations and then left to generalize — a brittle strategy on long-tail scenarios like unusual intersections or rare merging behavior. R2LPL closes that loop: the car's failures become its curriculum. Tested on the large-scale nuPlan closed-loop benchmark, including the deliberately punishing Test14-hard split, the system lifted a mediocre baseline planner to state-of-the-art performance within a handful of training cycles.
The bottleneck R2LPL targets is a real one. Closed-loop testing reveals where a policy fails, but raw failure data does not tell the model what it should have done instead. By filtering for recoverable mistakes — situations where a correct action still existed — and retrieving feasible corrective targets, R2LPL converts sparse failure evidence into compact, usable training signal. That is a meaningful distinction from simply replaying crashes at the model.
The broader AV industry has leaned heavily on fleet-scale data collection and human-labeled edge cases to patch policy weaknesses, a process that is expensive and slow. A self-correcting loop that requires fewer rollout cycles to hit competitive benchmarks would reduce that dependency — if it holds outside controlled evaluation settings, which remains the standard caveat for any paper that has not yet met a real parking lot.