A new dataset exposes a quiet blind spot in the natural-language-to-database market: language models that ace SQL benchmarks fall apart on MongoDB.
Researchers built TEND - Text-to-NoSQL Dataset - a benchmark of 1,210 MongoDB aggregation pipeline tasks spanning 11 databases. Unlike prior work that effectively dressed SQL patterns in MongoDB Query Language syntax, TEND was designed from scratch for document stores. Domain experts hand-defined collection boundaries, nested arrays, optional fields, polymorphic document shapes, and dynamic keys, then populated each database with real data and verified every query against a frozen MongoDB instance. The team also introduced SAG, a solver that reads actual stored documents to ground query paths and values before generating MongoDB queries. Standard LLMs - including models that rank well on SQL tasks - degraded substantially when run through TEND.
The gap matters because most of the "chat with your database" tooling currently shipping assumes relational tables with fixed schemas. Document databases like MongoDB power a large share of production infrastructure, and their nested, schema-less structure breaks assumptions baked into SQL-trained models. By building an execution-verified benchmark that tests real document reasoning rather than SQL transfer, the research gives the field something it has lacked: an honest measuring stick.
Any enterprise running MongoDB and eyeing an AI query layer should know that a strong SQL benchmark score is not a credential for this job.