AI agents struggle when Earth-science data gets complicated.
Researchers have released TerraBench, a benchmark designed to test whether AI agents can reason across the kinds of heterogeneous inputs that real environmental science demands — satellite imagery, gridded physical data, geospatial layers, and simulator outputs all at once. Built on TerraAgent, a ReAct-style framework that interleaves language model planning with scientific tool calls, the benchmark covers 403 tasks spread across three tracks and eight application domains, with 24,500 verified execution steps. It also introduces process-level tool-use metrics alongside tolerance-aware numeric scoring, a first for this space.
The gap TerraBench targets is real: climate foundation models can forecast, but they do not reason interactively in language; LLMs reason in language, but choke on high-dimensional Earth-system data. Prior benchmarks tested these capabilities in isolation. TerraBench puts them together, which is closer to what a working climate scientist actually faces. The results suggest that handing an agent a toolbox is not enough — it also has to coordinate heterogeneous workflows, parameterize tools precisely, and maintain artifact provenance.
The benchmark arrives as labs race to claim scientific credibility for general-purpose agents. Whether any current model meets the bar TerraBench sets is, diplomatically, an open question.