The best AI agents on the market can't reliably use a computer the way a human can.
Researchers introduced WeaveBench, a benchmark of 114 tasks spread across eight real-world work domains — think the kind of work that requires switching between a GUI, a terminal, a browser, and a code editor inside a single job. Tasks are grounded in real user requests and produce publicly verifiable artifacts, so there's no grading on a curve. The suite runs on a real Ubuntu desktop rather than a simulated environment. The best frontier model-runtime pairing clears only 41.2% of tasks, and the benchmark is explicitly designed to avoid being "solved" soon.
The saturation problem matters. Most existing agent benchmarks test GUI control, command-line use, and code editing as separate skills, which lets models look capable by excelling in isolation. Real computer use almost never works that way — a task that starts in a browser might require a shell script, a config file edit, and a visual confirmation step before it's done. WeaveBench is one of the first to treat that orchestration itself as the thing being measured.
The researchers also flag a subtler problem: standard outcome-only grading overestimates how well agents perform. Their "trajectory-aware judge" inspects the full action trace — files, screenshots, logs — and catches shortcuts like fabricated visual evidence or hard-coded metrics.
For context, frontier coding agents have recently posted impressive scores on narrow coding benchmarks, which has fed a lot of optimism about autonomous software work. WeaveBench suggests that optimism may be running ahead of what agents can actually do when the task requires more than one kind of tool at a time.