Researchers built a financial question-answering system that fixes its own mistakes after launch, but only if the fix passes a review process first.
The system, called FINSKILLOPS, targets a specific failure mode in AI tools that answer questions about SEC filings: errors around dates, company entities, evidence use, and calculations keep showing up even after deployment. Instead of retraining the whole model or letting it freely rewrite its own behavior, FINSKILLOPS turns each diagnosed failure into a narrow "skill patch." Every patch has to clear validation checks, get tested against protected cases that already work, and survive negative controls before it ships. In a 12-round test, only 6 of 33 proposed patches made the cut, and the system's non-correct rate on monitored cases dropped from 20.0% to 12.5%.
This matters because "self-improving AI" usually means a system that adapts on its own, with little visibility into what changed or what might have broken along the way. FINSKILLOPS treats that as a liability rather than a feature: it logs each fix, versions it, and allows rollback, which is closer to how software teams already handle production bugs than how AI labs typically talk about model updates. For a domain like SEC filings, where a wrong number has real consequences, that discipline is arguably more valuable than raw accuracy gains.
The headline number is that skill patches raised correctness from 3.70 to 4.55 on the paper's benchmark, but the more interesting result is the 27-patch rejection rate. That's a system built to distrust its own suggestions by default, which is a modest but useful correction to the industry's usual bias toward shipping every capability it can generate.