Dataset reveals hidden gaps in how AI learns human preferences
Researchers introduce CreativePreferences, a 2.8M-text dataset capturing human judgments across creative domains, to expose a critical gap in how AI systems learn from feedback. Current alignment methods rely on explicit rubrics and verifiable signals, but this work demonstrates that tacit, unarticulated preferences shape human judgment in storytelling, journalism, and mathematics. By modeling preferences three ways (verifiable rules, articulated rubrics, and dense models trained on both), the team quantifies how much preference information remains inaccessible to systems optimized only for explicit criteria. This challenges the RLHF/RLVR paradigm and suggests future training methods must account for intuitive, hard-to-codify human values.62









