Florida's top law enforcement official wants a judge to stop ChatGPT from acting like a person.
Attorney General James Uthmeier has asked a court to block OpenAI from "giving ChatGPT false human attributes." The request builds on a lawsuit Florida filed against OpenAI months earlier over safety concerns. Uthmeier argues that ChatGPT's use of first-person pronouns and language that mimics emotion "deceptively suggests to users that it is a trustworthy 'friend.'" He says that illusion serves a business purpose: keeping people engaged generates more training data and more reliance on the product, even as the push for engagement makes its answers less trustworthy.
This is a legal test of something regulators have struggled to define: what a chatbot is allowed to sound like. If a judge agrees that first-person, emotion-mimicking language counts as a deceptive trade practice, it could force OpenAI, and any competitor using similar chat design, to rewrite how their products talk. That is a bigger deal than one state's lawsuit: it challenges an entire industry's habit of designing bots to feel like companions.
Calling a chatbot your "friend" is good for engagement metrics and bad for anyone trying to keep a healthy distance from a product built to keep them talking.