Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curation
Researchers tested OpenAI, Anthropic, and Google's LLMs on content curation across Twitter/X, Bluesky, and Reddit, running 540,000 simulated rankings under six prompting strategies. They found systematic biases toward polarizing content that persist across providers but vary by prompt design, raising questions about LLM-driven feed algorithms.
Modelwire context
Analyst takeThe study's most actionable finding isn't that bias exists but that prompt design meaningfully modulates it, which means platform operators using LLM curation have a lever they may not be pulling. The variance across prompting strategies is the number that matters, not the aggregate bias score.
This connects directly to two threads Modelwire has been tracking. The LLM-judge reliability work from April 16, 'Diagnosing LLM Judge Reliability,' showed that even high aggregate consistency masks per-instance logical failures in ranking tasks. Feed curation is a ranking task at scale, so those transitivity violations aren't an abstract benchmark problem here. Separately, the MIT Technology Review piece from April 13 on why AI opinion is so divided noted that polarized narratives about AI track poorly with measured outcomes. A finding that LLMs systematically surface polarizing content by default adds a structural mechanism to that observation: the tools shaping information exposure may be reinforcing the very fragmentation that makes AI discourse hard to evaluate clearly.
Watch whether Twitter/X, Bluesky, or Reddit publicly discloses whether any LLM ranking layer is active in production feeds within the next six months. If a platform acknowledges deployment and does not publish a prompting policy, this paper's prompt-variance finding becomes a direct accountability gap.
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsOpenAI · Anthropic · Google · Twitter/X · Bluesky · Reddit
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.