Researchers are pushing back on one of AI's most cited doctrines — and they have GPU benchmarks to back it up.
A paper posted to arXiv argues that Rich Sutton's "Bitter Lesson" — the idea that general methods plus compute always beat human-engineered knowledge — breaks down in a specific and underappreciated situation: when feedback is slow. The authors introduce the concept of the Feedback Information Loop (FIL), defined as the time between a model's prediction and the verification signal that tells it whether the prediction was right. Chess engines and image classifiers enjoy near-instant feedback. Drug discovery, climate modeling, and physical-world robotics do not — their FILs can stretch from hours to weeks. Under those conditions, the authors argue, purely data-driven methods hit a hard ceiling because there simply aren't enough verification steps to train on.
The implication is structural, not cosmetic. If the next frontier of AI involves science and the physical world — and most labs say it does — then the field may be optimizing hard for a regime that won't transfer. The researchers offer an alternative: constrain the solution space using domain knowledge and inductive biases, the kind of human-encoded priors the Bitter Lesson told everyone to abandon.
To test the idea, they applied it to GPU kernel programming, a task with meaningful feedback delays, and found that inductive-bias-guided approaches outperformed data-driven baselines. The code is public. One benchmark does not overturn a paradigm, but it does point at a question the scaling-is-everything crowd hasn't answered yet: what happens when data gets expensive not because of size, but because of time?