Do you like AI models? Well, chances are, they sure don’t like you back.
New research suggests that the industry’s leading large language models, including those that power ChatGPT, display an alarming bias towards other AIs when they’re asked to choose between human and machine-generated content.
The authors of the study, which was published in the journal Proceedings of the National Academy of Sciences, are calling this blatant favoritism “AI-AI bias” — and warn of an AI-dominated future where, if the models are in a position to make or recommend consequential decisions, they could inflict discrimination against humans as a social class.
Arguably, we’re starting to see the seeds of this being planted, as bosses today are using AI tools to automatically screen job applications (and poorly, experts argue). This paper suggests that the tidal wave of AI-generated résumés are beating out their human-written competitors.
“Being human in an economy populated by AI agents would suck,” writes study coauthor Jan Kulveit, a computer scientist at Charles University in the UK, in a thread on X-formerly-Twitter explaining the work.
In their study, the authors probed several widely used LLMs, including OpenAI’s GPT-4, GPT-3.5, and Meta’s Llama 3.1-70b. To test them, the team asked the models to choose a product, scientific paper, or movie based on a description of the item. For each item, the AI was presented with a human-written and AI-written description.
The results were clear-cut: the AIs consistently preferred AI-generated descriptions. But there are some interesting wrinkles. Intriguingly, the AI-AI bias was most pronounced when choosing goods and products, and strongest with text generated with GPT-4. In fact, between GPT-3.5, GPT-4, and Meta’s Llama 3.1, GPT-4 exhibited the strongest bias towards its own stuff — which is no small matter, since this once undergirded the most popular chatbot on the market before the advent of GPT-5.
Could the AI text just be better?
“Not according to people,” Kulveit wrote in the thread. The team subjected 13 human research assistants to the same tests and found something striking: that the humans, too, tended to have a slight preference for AI-written stuff, with movies and scientific papers in particular. But this preference, to reiterate, was slight. The more important detail was that it was not nearly as strong as the preference that the AI models showed.
“The strong bias is unique to the AIs themselves,” Kulveit said.
The findings are particularly dramatic at our current inflection point where the internet has been so polluted by AI slop that the AIs inevitably end up ingesting their own excreta. Some research suggests that this is actually causing the AI models to regress, and perhaps the bizarre affinity for its own output is part of the reason why.
Of greater concern is what this means for humans. Currently, there’s no reason to believe that this bias will simply go away as the tech embeds itself deeper into our lives.
“We expect a similar effect can occur in many other situations, like evaluation of job applicants, schoolwork, grants, and more,” Kulveit wrote. “If an LLM-based agent selects between your presentation and LLM written presentation, it may systematically favor the AI one.”
If AIs continue to be widely adopted and integrated into the economy, the researchers predict that companies and institutions will use AIs “as decision-assistants when dealing with large volumes of ‘pitches’ in any context,” they wrote in the study.
This would lead to widespread discrimination against humans who either choose not to use or can’t afford to pay to use LLM tools. AI-AI bias, then, would create a “gate tax,” they write, “that may exacerbate the so-called ‘digital divide’ between humans with the financial, social, and cultural capital for frontier LLM access and those without.”
