A new metric aims to catch recommendation algorithms that fake "serendipity" by tossing in random junk nobody wants.
The paper, published this week, introduces SPADE (Serendipitous Pareto Distance Evaluation). It plots every item in a two-dimensional space defined by popularity and similarity to a user's history, then calculates a personalized Pareto frontier of the most popular, most similar items for that user. A recommendation's serendipity score is the distance of correctly predicted test-set items from that frontier: farther away, while still relevant, scores higher. The authors tested SPADE across five datasets and five baseline recommendation algorithms and found it held up.
Older "beyond-accuracy" metrics tend to reward novelty or popularity in isolation, which lets an algorithm score well by recommending obscure or random items nobody actually wants. SPADE ties its score to whether the recommendation was a genuine hit in the test set, closing that loophole. That distinction matters for any platform, from retail to streaming to news, trying to tell whether its algorithm helps people find something they'll like, or just something different.
Still, this is a benchmark result on five offline datasets, not a live test on real users scrolling a feed. Whether SPADE-optimized picks actually feel serendipitous to a human is a separate question nobody has answered yet.