[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-why-asking-people-to-choose-may-break-ai-alignment":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},3648,"why-asking-people-to-choose-may-break-ai-alignment","Why Asking People to Choose May Break AI Alignment","A new paper argues that forced pairwise comparisons, a core tool in AI alignment, fail when people hold multiple conflicting values at once.","Forcing people to pick a winner in every comparison may be quietly corrupting the data AI alignment relies on.\n\nResearchers publishing on arXiv have built a formal model of what they call \"internal pluralism\" — the idea that a single person can hold multiple, equally authoritative priorities about how an automated decision rule should behave. Things like proportionality, egalitarianism, and equal treatment don't always point the same direction. The paper identifies two specific failure modes. First, priorities that are inherently global — what they require in one case depends on outcomes elsewhere — can't be captured by local, one-on-one comparisons. Second, when two strongly-held priorities conflict, forcing a choice produces behavioral distortions that contaminate the preference signal.\n\nThis matters because pairwise comparison is the backbone of human preference learning in alignment work, including the reinforcement learning from human feedback pipelines that train most large language models today. If the inputs are systematically distorted, the resulting models inherit those distortions — and the paper suggests this isn't a fringe edge case but a structural problem baked into the method. The fix the researchers propose — letting respondents report indecision rather than forcing a pick — turns out to dramatically reduce the number of queries needed to learn preferences accurately.\n\nThe field has spent years debating whether RLHF annotators are consistent or well-calibrated; this paper reframes the question by suggesting the format itself is the problem, not the annotators.","[\"ai\",\"alignment\",\"machine-learning\",\"research\"]","2026-07-07T04:00:00.000Z","2026-07-07T04:45:49.098Z","2026-07-07T04:45:52.038Z","published",null,[],"ai",[24,26,27,28],"alignment","machine-learning","research",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.02672",0,{"sections":35},[36,40,45,50,55,60,65,70,75,79,84,88,93,98],{"name":37,"slug":24,"count":38,"latest_published_at":39},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":41,"slug":42,"count":43,"latest_published_at":44},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":46,"slug":47,"count":48,"latest_published_at":49},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":18},"Dev Tools","dev-tools",59,{"name":80,"slug":81,"count":82,"latest_published_at":83},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":85,"slug":86,"count":82,"latest_published_at":87},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":89,"slug":90,"count":91,"latest_published_at":92},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":94,"slug":95,"count":96,"latest_published_at":97},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":99,"slug":100,"count":101,"latest_published_at":102},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]