A new study finds that how a chatbot personalizes itself - by remembering what you tell it, or by quizzing you upfront - shapes how much you open up, and how much you later regret it.
Researchers recruited 992 people for five days of daily advice-seeking conversations with language models, comparing a non-personalized baseline against two personalization methods: one that drew on the memory of prior conversations, and one built from a one-time intake survey. They tracked self-disclosure, perceived creepiness, and regret over what participants shared. Many changes in behavior over the five days turned out to be driven simply by repeated use, not personalization itself. But the personalization method still mattered: people in the memory-based condition shared more personal information and rated the AI as less creepy, while people in the survey-based condition reported more regret about what they had disclosed.
That distinction matters because 'AI that remembers you' is becoming a default feature, not an opt-in extra, across major chatbot products. This study is among the first to track what that actually does to users' disclosure habits and trust beyond a single session, rather than just measuring whether people liked the output.
It is a useful reminder that not all personalization is equal. A system that learns from what you say organically appears to build more comfort than one that interrogates you once and never asks again - a distinction product teams chasing 'personalized AI' as a checkbox feature would do well to notice.