Polarization by Default: Auditing Recommendation Bias in LLM-Based Content Curation

The study's most actionable finding isn't that bias exists but that prompt design meaningfully modulates it, which means platform operators using LLM curation have a lever they may not be pulling. The variance across prompting strategies is the number that matters, not the aggregate bias score.

This connects directly to two threads Modelwire has been tracking. The LLM-judge reliability work from April 16, 'Diagnosing LLM Judge Reliability,' showed that even high aggregate consistency masks per-instance logical failures in ranking tasks. Feed curation is a ranking task at scale, so those transitivity violations aren't an abstract benchmark problem here. Separately, the MIT Technology Review piece from April 13 on why AI opinion is so divided noted that polarized narratives about AI track poorly with measured outcomes. A finding that LLMs systematically surface polarizing content by default adds a structural mechanism to that observation: the tools shaping information exposure may be reinforcing the very fragmentation that makes AI discourse hard to evaluate clearly.

Watch whether Twitter/X, Bluesky, or Reddit publicly discloses whether any LLM ranking layer is active in production feeds within the next six months. If a platform acknowledges deployment and does not publish a prompting policy, this paper's prompt-variance finding becomes a direct accountability gap.