AWS has released UAEF, a free open-source tool that checks whether a change to an AI agent made it better or worse.
The tool scores agents on 36 measures, ranging from how well they use tools to safety. It works across several agent frameworks, the libraries developers use to connect a model to tools and tasks. It also flags a regression when a new prompt or a new model leaves an agent performing worse than before. The code is free to use.
The idea is not new. Software teams have long run regression tests, which rerun a fixed set of checks after every change so a fix in one place does not quietly break another. UAEF applies that habit to agents, where the two most routine edits, rewriting a prompt and swapping in a newer model, are exactly the changes that can shift behavior without anyone noticing.
The cross-framework part is the more interesting design choice. A test suite tied to one framework ties its results to that framework, which makes comparisons between setups awkward. A neutral scorer lets a team judge a prompt change or a model swap on the same 36 measures wherever the agent was built.
The open question is how those measures hold up. Tool use and safety, the two categories AWS names, are very different things to score. One can be checked against what the agent actually did, while the other leans on judgment calls about what counts as unsafe. How UAEF handles that spread decides whether a flagged regression is a signal worth acting on or just noise. The 36-measure description comes from AWS itself, so outside teams running it on their own agents will be the first real test.