A new paper argues that AI recommendation chatbots need to see what you see, not just hear what you say.
Posted to arXiv on September 17, 2026 as arXiv:2609.18249 (https://arxiv.org/abs/2609.18249), the paper introduces Re2A, a framework for what its authors call situated conversational recommendation. The idea: when a recommender assistant and a user share a physical space, like a store or a room, the assistant should reason over the conversation, the scene both parties can see, and the attributes of items in that scene together, rather than treating them as separate inputs. Re2A uses what the paper calls rubric-based preference reasoning to force the model to produce an explicit statement of the user's preferences, then applies a preference-conditioned optimization step so responses satisfy both what the user wants and what is actually available in the scene. The code is public on GitHub under developer DongdingLin's account (github.com/DongdingLin/Re2A), and the paper reports Re2A outperforming prior state-of-the-art methods on two situated conversational recommendation datasets.
This matters because most recommendation chatbots today, including the shopping assistants bolted onto retail apps, work from text alone: what you typed, what's in your cart history. Real stores, smart glasses, and AR shopping all involve a camera feed the assistant could also be reading. Re2A is a step toward recommenders that can say "the blue one, second shelf" instead of listing options with no idea what's actually in front of you.
Whether rubric-based reasoning holds up past two benchmark datasets in messy real-world scenes is a question the paper leaves open.