Researchers built an LLM agent that pulls satellite imagery from cloud catalogues using plain English, then spent time systematically trying to break it.
The system uses three coordinated agents: one interprets user intent, one translates that into schema-aware API calls, and a third acts as a safety guardrail to block policy violations. Together they make remote sensing datasets accessible without requiring users to understand the underlying API structure. Testing covered environmental monitoring and disaster response applications, drawing on satellite imagery and climate datasets. Adversarial testing used multi-turn prompts crafted to steer the system toward unintended API calls.
The failure mode that should concern practitioners here is not hallucination but API manipulation. When a system's output is a structured call against live geospatial infrastructure, a successful adversarial nudge is not a wrong answer on a quiz. It is a potentially unauthorized request against data services that underpin emergency response and climate analysis. Prompt-level safety instructions improved robustness in testing, but the paper found that some high-stakes failures persisted specifically in API manipulation scenarios.
The researchers propose a dedicated Guardrail agent as the fix, while noting the need to balance safety, usability, and cost. That is the same argument every security team has with every product team about how hard to gate a production API, and it rarely resolves cleanly.