A new crossover benchmark says the smart move for most tabular-data prediction problems is still to train a classical model, not prompt an LLM.
Researchers tested small GPT models against six families of classical machine-learning models across 18 tabular datasets, running 126 independent evaluations under eight different prompting setups. They measured a labeled-data crossover point - where a trained model's accuracy overtakes a frozen LLM working from a plain-English prompt with no training at all. Even giving the LLM its best possible prompt, a trained classical model won using no more data than a typical business already has on hand in 86% of cases. The crossover landed at a median of about 6% of the full training set, and in 40% of cases a trained model won using the smallest data sample tested.
That's a useful reality check against the current pitch for AI-in-a-spreadsheet features like Microsoft Copilot in Excel and Claude for Excel, which promise you can skip data collection and labeling entirely. The researchers also found that stuffing more few-shot examples into a prompt doesn't behave like real training - error rates don't follow the predictable power-law improvement that classical models show as they see more labeled rows.
None of this makes LLMs useless for spreadsheets - zero-shot labeling still earns its keep when you have no data and no time. But for a recurring business prediction problem, the paper's advice is blunt: collect a few hundred labels and train a gradient-boosted model instead of leaning on a chatbot.