A new reproducibility study says recommender systems that explain themselves in plain English work about as well as advertised - but the "edit your profile to fix bad recommendations" pitch doesn't hold up under scrutiny.
The paper reruns User Profile Recommendation (UPR), a method that turns a user's raw reviews on Amazon Movies & TV and TripAdvisor into a natural-language summary of their tastes, then uses that summary to rerank recommendations. The researchers confirm UPR's original claim: it matches competitive accuracy on a test-set reranking task while giving users a readable profile they can, in theory, correct. They go further than the original authors, running the model across five random seeds to check the results aren't a fluke, ablating pieces of the input context, and using the nnsight interpretability toolkit to probe what happens inside the model when a profile is deliberately altered.
That last test is the interesting part. Rewriting a user's profile does move the model's predicted ratings, but it moves them the same amount across every genre rather than boosting the genres the edit actually mentioned - so the ranked list of recommendations barely changes, even when researchers directly steered the model's internal activations. The authors trace this to the rating-regression objective UPR is trained on, not to the natural-language interface itself; models trained with a ranking objective instead handle the task noticeably better.
In other words, the readable profile is a nice window to look through, but right now it's mostly decorative - turning a knob behind that window barely moves the recommendations you actually see.