Researchers have built a tool that lets you ask a reinforcement learning agent what would have happened if it had done something else.
The library is called EARL, short for Explanations using Alternative Realities for Reinforcement Learning. It generates counterfactual, what-if explanations for RL agents by comparing an action the agent actually took against alternatives and showing the likely outcomes. That matters because deep RL policies run on neural networks, and neural networks do not explain themselves. Researchers say counterfactual explanations work well in psychology and are intuitive for people, but past attempts to bring them into RL mostly stayed stuck in toy examples and lab benchmarks. To show EARL works outside a benchmark, the team tested it on a simulation of CitiBikes, a self-adaptive bike-sharing system that has to keep rebalancing bikes across stations on its own.
Self-adaptive systems like bike-sharing networks, traffic signals, and cloud autoscalers increasingly hand decisions to RL agents that no one can fully audit. That opacity makes it harder to trust the system and harder to verify it is doing the right thing for the right reason. A tool that answers what-if questions on a realistic system, not just a toy grid world, is a step toward explanations engineers can actually use before shipping these agents into production.
A bike-sharing simulation is a reasonably clean test case with clear costs and clear failure modes. Whether counterfactual explanations hold up on messier, higher-stakes systems, and whether a plausible-sounding what-if actually counts as an explanation anyone should trust, is the harder question this demo does not answer.