A prototype AI assistant that turns your inline edits into permanent policies made people trust it more, according to a study posted to arXiv this week (arXiv:2509.12626).
The system, called DoubleAgents, pairs a coordination agent that drafts plans and actions with a dashboard that shows its reasoning and a policy module that converts user corrections into reusable rules - coordination policies, email templates, and "stop hooks" that block bad actions going forward. Researchers tested it in a two-day in-lab simulation with 10 participants, plus three real-world deployments and a separate technical evaluation. Over time, participants grew more comfortable delegating tasks and relied on the system more, a trend the paper ties directly to those three design pieces: visible reasoning, editable plans, and rules that stick. People still wanted to step in manually at edge cases and context-dependent decisions the agent couldn't fully judge on its own.
The finding matters because most "agentic AI" pitches sell autonomy first and trust second, assuming users will hand over tasks once the model gets smarter. This study flips that: trust grew because the system made its logic visible and let corrections accumulate into policy, not because the underlying model improved. That is a concrete, testable alternative to the usual black-box pitch dominating agent demos.
Ten participants in a lab simulation is not a verdict on how agents should work at scale - it is a hypothesis with early support and three case studies behind it.