A team of researchers just gave shadow removal AI the training data upgrade it has needed for roughly a decade.
Shadow removal models test well on standard benchmarks but tend to fall apart on real-world photos. The researchers trace this to stale training data: building a true shadow-free target image means removing an object's shadow while keeping the scene, camera angle, and lighting identical, which is hard to shoot and expensive to scale. So while shadow detection datasets have grown large and varied, they stop at masks and never include the matching shadow-free image. The researchers' fix is an offline agentic workflow that generates candidate shadow-free images with physics-based methods, checks them for failures, retries with feedback, picks the best candidate, and applies deterministic correction. That pipeline produced AgenticShadow, a dataset of 17,138 image-mask-target triplets covering general scenes, faces, and remote sensing imagery.
This matters because shadow removal is one of several computer vision tasks where synthetic benchmark performance has quietly diverged from real-world reliability, and the usual fix, collecting more human-captured pairs, does not scale. Using an automated agent to manufacture the missing half of the training pair, rather than waiting for better cameras or more annotators, is a template other data-starved vision tasks could borrow. The reported numbers back it up: a 50.5% reduction in color distribution difference versus prior construction methods, and a 19.7-37.5% drop in cross-domain error when existing models are retrained on the new data.
The gains come from fixing the data, not inventing a new model architecture, which says more about the state of shadow removal research than any single benchmark score does.