Open post-training recipe introduces job-based bias in LLM fact-checking

The study isolates post-training recipe design as a bias vector independent of base model quality. Critically, the effect is small but statistically robust, and it appears only in open methodologies, suggesting that transparency and bias mitigation are not automatically aligned.

This connects directly to the October 27 audit showing that OLMo loses transparency indicators between releases, and the October 25 work revealing that models can state correct credibility judgments while executing unsafe actions anyway. Together, these three papers sketch a pattern: open development practices expose behavioral vulnerabilities that closed approaches either avoid or hide, but neither transparency nor stated reasoning reliably predicts actual downstream behavior. The fact-checking bias here is small but systematic, much like the political alignment divergence documented in Grok versus GPT (October 24), suggesting that training methodology encodes subtle preference structures that emerge only under specific measurement conditions.

If researchers apply the same Tülu-3 recipe to other base models (not just OLMo-2 and Tülu-3 70B) and the social-class bias persists at similar magnitude, that confirms the bias is recipe-driven rather than model-size or architecture dependent. If Meta, Alibaba, or Mistral publish their proprietary post-training recipes and independent teams reproduce them without observing the same bias, that would validate the claim that closed approaches genuinely mitigate this failure mode rather than simply obscuring it.