Most "thumbs up or thumbs down" buttons on AI products are badly designed, according to new research built with eBay.
The study, a multi-phase collaboration between academic researchers and eBay, started by benchmarking how current industry products collect user feedback on generative AI features. The team found recurring problems: feedback options were hard to discover, the terminology used was unclear to users, and the mechanisms rarely considered what users actually get out of giving feedback. From those findings, the researchers wrote best-practice recommendations and built a prototype feedback-collection tool, which they then tested. The goal was a tool that felt efficient and positive to use for everyday users while handing product teams usable, detailed data on performance and problems.
This matters because governments and industry guidelines already require post-deployment monitoring of AI systems, but offer little guidance on how to actually build feedback mechanisms people will use. Feedback is one of the cheapest ways to catch an AI product's failures at scale, but only if anyone bothers to click the button and the input means something.
If your company has shipped a feedback form nobody uses, this is a reminder that collecting data isn't the same as learning from it.