A new academic benchmark reveals how far AI terminal agents still have to go.
Researchers introduced TUA-Bench, a 120-task benchmark designed to evaluate AI agents working inside a command-line terminal on general-purpose work — not just coding. The tasks span five families: document editing, email management, live-web searches, and scientific and engineering workflows built with input from PhD-level domain experts. Each task runs in a real terminal environment with a deterministic setup script and is scored by whether the agent actually completes it, not by how it looks. The strongest result came from Claude Code running Claude Opus 4.8 at maximum reasoning effort, which cleared 65.8% of tasks overall.
The gap that benchmark exposes matters because most existing evaluations push AI agents in one of two narrow directions: graphical interfaces or shell-heavy programming tasks. TUA-Bench deliberately targets the middle ground — the routine digital work that happens in a terminal but has nothing to do with writing code. A 65.8% ceiling from the leading model means roughly one in three tasks still fails, and the researchers note substantial gaps across both the general-use and specialist tracks.
The benchmark's arrival follows a stretch of heavy investment in so-called computer-use agents, with Anthropic, OpenAI, and Google all shipping variations on the idea that AI can sit at a keyboard and get things done. The results here suggest the keyboard is easier to hand over in a graphical window than in a shell — which is a useful corrective to the promotional language that has surrounded these launches.
A 65.8% pass rate sounds respectable until you remember that a human intern clears most of these tasks on the first try.