A research team has built a training pipeline that makes vision-language models both more accurate and dramatically faster for autonomous driving — without external tools or verbose chain-of-thought prompting.
The framework, called CritiqueDriveVLM, works in three stages. First, a reinforcement learning process uses a multi-dimensional verifier to give the model granular scalar feedback across multiple turns, forcing it to internalize logical reasoning rather than rely on external scaffolding. That produces what the researchers call a System-2 Teacher — a model that reasons carefully and achieves 76.54% on the DriveLMM-01 multiple choice benchmark, up from a 55.54% baseline. The team then applies "latent thought distillation," aligning a leaner System-1 Student model's internal representations to match the Teacher's fully converged reasoning states. The Student inherits the reasoning depth without the verbosity, generating an average of just 28 tokens per response.
The latency drop is where this matters for real deployment: inference time falls from 3,482 ms to 416 ms — an 88% reduction. That gap between a model that reasons well in a lab and one that can actually operate in a moving vehicle has been the central blocker for end-to-end VLM adoption in autonomy stacks. A system that needs three and a half seconds to think is not driving anything safely.
Most chain-of-thought and tool-augmented approaches trade speed for accuracy; this work claims to keep accuracy gains while nearly eliminating the latency penalty. The distillation idea borrows from established model compression research, but applying it specifically to compress reasoning states — not just output behavior — into a faster student is the novel move. Whether the benchmark improvements hold on closed, real-world driving datasets remains the open question every autonomy paper eventually has to answer.