A new study finds that tabular foundation models outperform conventional ML when labeled examples are scarce, but lose ground as labels accumulate.
Researchers tested three foundation models and several conventional machine learning models on three real crowd-monitoring datasets from the Hajj and Umrah pilgrimages. Foundation models, which generate predictions from a small number of examples without task-specific training, held a clear accuracy edge when labels were very few. As the label count rose, tuned conventional models caught up and pulled ahead, especially on a structural geometry classification target. On efficiency, foundation models skip tuning costs but reprocess their entire context at each prediction, while tuned conventional models pay a large upfront tuning cost and then predict cheaply.
The finding matters because most benchmark comparisons assume ample labeled data. This study draws a practical line: below a certain label budget, you reach for a foundation model; above it, you tune a conventional model. For any team working in domains where labeling is expensive, that crossover point is worth knowing before the labeling work starts.
The paper's output is a decision map, not a champion. That kind of practical, unglamorous result is harder to hype but easier to deploy.