Open-weight AI models increasingly used to screen job candidates can be nudged toward bias by nothing more than the wording of a job listing, according to a new academic audit.
Researchers tested six open-weight models - Llama 3.2, Mistral, Gemma 3, Qwen 3, Phi 3, and DeepSeek-R1 - across four experiments that simulated both recruiters screening resumes and job seekers deciding whether to apply. Job postings written in assertive, "agentic" language lowered recommendation scores for female candidates, while more "communal" phrasing partly reversed that penalty. Postings containing coded-exclusion language had a bigger effect still, suppressing scores for non-White candidates and making non-White personas less likely to say they would apply at all. A follow-up test found that explicitly labeling a candidate's demographic identity was the single biggest driver of the skew, and separate word-embedding bias tests matched the pattern.
The researchers frame the stakes around specific regulation: the EU AI Act classifies hiring tools as high-risk systems requiring documented bias testing, and the US EEOC's adverse-impact rules can apply if a tool disproportionately screens out a protected group. That gives employers deploying these models a concrete compliance reason to check for this kind of language-driven skew, not just an ethical one. The paper proposes a pre-deployment audit protocol - scoring posting language, probing models with different candidate personas, and flagging adverse impact against the standard four-fifths threshold - as one way to catch it before a model goes live.
It's a worthwhile reminder that "bias in the model" is only half the story. The words a company puts in a job ad can do damage well before any AI gets involved.