A new paper proposes a way to make AI chatbots quietly learn your preferences without ever asking you to rate anything.
Researchers describe a framework called COPE (Continual Optimization with Personalized embedding and self-Evaluation) that assigns each user a learnable embedding, essentially a private profile the model updates over time. Instead of relying on thumbs-up or thumbs-down clicks, which most users never bother to give, COPE has the model evaluate its own responses and generate a proxy reward signal to guide updates. The system folds preference capture, self-evaluation, and response optimization into a single training step, and the researchers say it keeps working alongside Retrieval-Augmented Prompting rather than replacing it. In testing, COPE beat both prompt-engineering approaches and other training-based personalization methods when explicit feedback was scarce.
Most personalization tricks today either eat up context window space with lengthy instructions or freeze after initial training, meaning they cannot adapt as someone's tastes shift. COPE's self-evaluation trick sidesteps the classic cold-start problem of personalization: you do not need users to rate outputs for the system to keep learning. That said, letting a model grade its own homework to generate a training signal is exactly the kind of shortcut that invites quiet drift if the self-evaluator's judgment is off.
The paper reports the self-evaluation stays reliable and general capabilities remain stable, but those are the authors' own benchmarks - real-world users, and their actual preferences, are messier than any test set.