AI/ ai agents · llm benchmarks · evolutionary algorithms · arxiv

RoboPhD evolves AI agent pipelines to beat rivals at every price

A new evolutionary AI system called RoboPhD assembles agent programs from nine priced LLM APIs to beat rivals' accuracy at their own price points.

RoboPhD, an evolutionary meta-agent, beat rival AI agents on both cost and accuracy at the same time, and it only needed up to 100 training examples per task to do it.

The system, described in a new arXiv paper, starts with a priced menu of nine LLM endpoints, brief documentation of a task and its API, and a simple seed agent. From there it evolves complete agent programs aimed at a specific per-problem cost target, usually set to match what an incumbent competitor already charges. The researchers ran it against two unrelated benchmarks: DS-1000, which checks generated code by actually executing it, and PaperFindingBench, which uses an LLM judge to score how well an agent retrieves relevant scientific papers.

On official leaderboard submissions for both benchmarks, RoboPhD's agents took every Pareto-frontier slot but one, meaning almost no other entry offered better accuracy for the same price, or a lower price for the same accuracy. That included beating both the highest-scoring competitor and the cheapest one, on their own turf. If a technique like this holds up outside arXiv leaderboards, it's a rough afternoon for anyone selling a thin agent wrapper around someone else's foundation model.

Worth remembering: this is a self-reported leaderboard result on two specific, narrow benchmarks, not an independent audit of general-purpose agents in production.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →