A technique for auditing neural network decisions by tracing them to individual training examples could make high-stakes AI deployments harder to hide behind.
Researchers published a method that decomposes what a neural network decides into a weighted sum of its training cases. The math relies on fitting a simple ordinary least squares probe on top of a frozen neural representation — no retraining required, no need to dig up the original training run. Each action score the network produces gets broken down by how much each training example contributed, with the weights determined by something called empirical Gram geometry. The team tested it on synthetic data, an energy market dataset, and two standard fairness benchmarks — Adult Income and Default Credit — and found it outperformed other attribution methods on a consistency metric called Top-30.
The stakes here are real. Regulators and auditors in credit, healthcare, and energy markets increasingly want case-level explanations: not just "the model said no" but which prior examples pushed it there and what outcomes those cases carried. Existing explainability tools tend to highlight input features, not training data — this approach flips the lens. If it holds up to scrutiny, it gives compliance teams something more concrete to work with than saliency maps and attention weights.
The method still depends on how well the OLS probe reconstructs the model's scores — if that fidelity is low, the training-case breakdown may be more narrative than evidence.