A research paper proposes two algorithms that make AI-powered SQL queries significantly cheaper to run.
Modern data warehouses increasingly embed large language models directly into SQL queries, running inference on every matching row — an approach that gets expensive fast. Model cascades address this by sending most rows through a cheap "proxy" model and escalating only ambiguous cases to a pricier "oracle" model. The problem: existing cascade methods (specifically the SUPG family) require a full pass over proxy scores before any results flow out, which blocks pipelined query engines and forces a tradeoff between precision and recall rather than optimizing both. The new paper formalizes this as a streaming problem and offers two solutions. SUPG-IT adapts the existing SUPG approach to streaming by iteratively tightening two routing thresholds as oracle labels arrive in batches — the first streaming cascade with joint probabilistic guarantees on both precision and recall simultaneously. GAMCAL takes a different angle: instead of asking users to specify targets, it learns a calibrated model that maps proxy scores to true-positive probabilities and routes stochastically based on a single cost-accuracy tradeoff parameter.
The efficiency gains matter because semantic SQL is no longer a research curiosity — production data warehouses are already running it, meaning inference costs are real and recurring. GAMCAL reached F1 above 0.95 while making 58% fewer oracle calls than the LOTUS SUPG baseline on the same benchmarks, which translates directly to API spend. SUPG-IT posted a mean F1 of 0.989 across six datasets, the highest best-case accuracy of the two.
None of this solves the deeper issue that stuffing LLM inference into SQL is architecturally odd — but given that it is happening regardless, smarter routing is a reasonable harm-reduction strategy.