A new paper proposes running a gold-standard uncertainty method on just the last layer of a neural network — and getting most of the benefit at a fraction of the cost.
Hamiltonian Monte Carlo has long been the benchmark for measuring how confident a neural network should be in its predictions. The catch: it's expensive. Running it across an entire large model is computationally prohibitive, which is why it rarely shows up outside of small research settings. The new approach, called Last Layer HMC (LL-HMC), sidesteps that by restricting the sampling to only the final layer of a deep neural network. The authors tested it against five competing probabilistic methods on three real-world video datasets focused on driver behavior.
The results are pragmatic rather than spectacular. LL-HMC matched rivals on in-distribution classification and out-of-distribution detection — the two things that matter most when you need a model to know what it doesn't know. Adding more samples improved OOD detection but didn't help classification, and running multiple chains produced no consistent gains.
For teams building safety-critical systems — autonomous driving being the obvious example given the datasets used — this is a useful trade-off: reasonable uncertainty estimates without the compute bill that full HMC demands. It won't replace full Bayesian inference where resources allow, but it gives practitioners a more deployable middle ground than most alternatives currently offer.