Automated accessibility testing just got a meaningful upgrade for the class of bugs it has always been bad at finding.
Researchers introduced Flow-A11y, a system that runs natural-language-described interaction scenarios in a live browser and captures a runtime trace of what actually happens. It then builds criterion-specific evidence packets before issuing any finding — meaning it won't flag a failure unless it can point to concrete runtime proof. Tested across 19 real public-web scenarios covering 45 dynamic WCAG criteria, the system achieved more than ten times higher agreement with human oracle judgments than a generic browser-agent audit. Its evidence-calibration layer pushed fail precision from 23.5% to 41.4% — nearly doubling the share of flagged failures that are genuine, not false alarms.
The gap it closes is real: keyboard traps, focus loss after modal close, and delayed status updates are the kind of bugs that only appear mid-interaction, so static scanners powered by DOM snapshots simply cannot see them. Getting even close to automating that class of defect matters for teams who currently rely on manual screen-reader walkthroughs, which are slow and inconsistently applied across release cycles.
Fail precision still sits below 50%, which means more than half of flagged dynamic failures would still need human review — so this is a triage aid, not a sign-off tool, and anyone treating it as the latter will regret it.