A new looped transformer claims to match a leading tabular AI model on benchmark tasks while using nearly 90% fewer parameters.
Researchers behind LoopICL built a looped transformer that reuses a single transformer block instead of stacking dozens of unique layers. The design splits input into two streams, one tracking individual cell values and one tracking whole in-context examples, and refines both through attention across rows and columns. A learned exit gate lets the model decide how many times to loop at inference, trading compute for accuracy on the fly. In the preprint, the authors report LoopICL matches TabICLv2, a leading in-context tabular model, on the TabArena and TALENT benchmarks at equal compute cost, using roughly 90% fewer parameters, though those figures come from the paper itself and haven't been independently confirmed on a public leaderboard.
If those numbers hold up, it's a real challenge to the assumption that tabular foundation models need huge parameter counts to beat gradient-boosted trees, the longtime standard for structured data. It also hands practitioners a dial they rarely get elsewhere: adjust compute up or down at inference without retraining, which matters more for spreadsheet-shaped data than for a flashy generative demo.
Whether LoopICL's compute-matched claims survive scrutiny will depend on a full peer review, an independent TabArena leaderboard entry, and released code, the usual gap between a preprint's abstract and a benchmark anyone else can run.