Running AI on big databases via cloud APIs can cost more than $10,000 per experiment — and a new paper argues that's unnecessary.
Researchers integrated quantized, open-weight language models into BlendSQL v0.1.0, a framework for LM-enhanced relational database queries. The models ran locally on 16GB of VRAM — consumer-grade hardware by today's standards — and matched or exceeded the accuracy of closed-source API alternatives while delivering a 390x cost reduction and 3.8x latency improvement. The work comes out of Capital One Research, and the code is public on GitHub.
The implications reach past cost savings. Proprietary API pricing has quietly shaped which database AI research gets done at all: experiments that cost five figures are experiments that only well-funded teams attempt. Cheaper local inference doesn't just save money — it changes who can run the experiments, which changes what gets built.
The open-weight moment has been building for a while, but most of the case studies so far have been about chatbots and code generation. Applying the same logic to data infrastructure — where query volumes are high and budgets are finite — is a more concrete argument for local models than most advocates have managed to make.