[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-llm-personality-scores-are-mostly-a-measurement-glitch":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},1680,"llm-personality-scores-are-mostly-a-measurement-glitch","LLM Personality Scores Are Mostly a Measurement Glitch","A new psychometric study finds that up to 90% of variation in AI personality test results reflects how models handle scale formatting, not actual traits.","Give an AI a personality test and you get a personality — just not a real one.\n\nResearchers administered a battery of personality and risk-preference instruments to 56 instruction-tuned large language models alongside large human reference groups. They found that 81-90% of the variation between models came not from the psychological traits the tests were designed to measure, but from what the paper calls directional response bias — a model's tendency to consistently favor one end of a scale or one labeled option regardless of the question being asked. In humans, that same bias accounts for only 9-16% of variation. The team also found that a model's apparent profile shifts depending on which items from a test are selected, meaning a profile can be manufactured simply through item choice.\n\nThis matters because LLM psychological profiles are not academic trivia. Labs, regulators, and researchers use them to make claims about model safety, usability, and suitability as stand-ins for human participants in studies. If those profiles are artifacts of the measurement instrument rather than properties of the model, then conclusions built on them — including some safety assessments — are on shaky ground.\n\nThe authors note the bias shrinks as model capability increases but does not disappear, and they call for purpose-built assessments centered on what they term response orthogonality: the share of test items where the target trait and the directional bias actually pull in opposite directions. In other words, the field has been borrowing a ruler designed for humans and expressing surprise when it keeps measuring something else.","[\"ai\",\"research\",\"llms\",\"safety\"]","2026-06-19T04:00:00.000Z","2026-06-19T09:49:17.742Z","2026-06-19T14:21:36.865Z","published",null,[],"ai",[24,26,27,28],"research","llms","safety",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.20205",0,{"sections":35},[36,39,43,48,53,58,63,67,71,76,81,86,91,96],{"name":37,"slug":24,"count":38,"latest_published_at":18},"AI",490,{"name":40,"slug":41,"count":42,"latest_published_at":18},"Security","security",132,{"name":44,"slug":45,"count":46,"latest_published_at":47},"Policy","policy",88,"2026-06-16T09:26:09.000Z",{"name":49,"slug":50,"count":51,"latest_published_at":52},"Consumer Tech","consumer-tech",78,"2026-06-16T17:58:24.000Z",{"name":54,"slug":55,"count":56,"latest_published_at":57},"Hardware","hardware",62,"2026-06-18T15:24:16.000Z",{"name":59,"slug":60,"count":61,"latest_published_at":62},"Deals","deals",58,"2026-06-19T14:43:50.000Z",{"name":64,"slug":65,"count":61,"latest_published_at":66},"Software","software","2026-06-16T20:00:00.000Z",{"name":68,"slug":69,"count":70,"latest_published_at":18},"Dev Tools","dev-tools",50,{"name":72,"slug":73,"count":74,"latest_published_at":75},"Science","science",38,"2026-06-18T04:00:00.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Gaming","gaming",31,"2026-06-16T15:25:13.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"General","general",26,"2026-06-13T18:35:15.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"Startups","startups",23,"2026-06-16T15:00:00.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"Reviews","reviews",19,"2026-06-14T08:00:00.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]