AI agents that ace clean benchmarks can fall apart the moment their environment gets messy.
A new paper describes Env-Rethink, a system built around a 27B post-trained model, aimed at large language model agents working in real-world settings where files are scattered, evidence conflicts, and conditions shift over time. The researchers found that these conditions can cut a top agent's performance from 83.9% to 57.6%. Env-Rethink builds "Collection Maps" to organize related files and "Event Logs" to track how pieces of information relate over time, then uses offline trajectory learning to spot recurring sources of noise. It also generates synthetic event histories that deliberately complicate the environment, producing harder training scenarios. The paper, posted to arXiv on September 25, 2026, reports the approach lifted rubric pass rates by more than 15.1 percent across nine different models on 30 tasks.
Office workflows and lab notebooks are not tidy sandbox benchmarks - they are full of duplicate drafts, stale documents, and conflicting instructions, exactly the mess most agent evaluations skip. That gap between benchmark and reality is a big reason agents that look sharp in demos stumble on real jobs. Env-Rethink's approach - training against the environment itself, not just the model - points at a fix that does not depend on simply building bigger models.
The 15.1 percent gain is measured on the nine models the researchers chose, not on whatever agent your IT department is about to license, so treat the number as a proof of concept rather than a warranty.