Does Self-Consistency Improve the Recall of Encyclopedic Knowledge?

The more interesting finding isn't the 89% headline number — it's the methodological move of splitting MMLU into reasoning versus recall subsets at all, which exposes how conflated that benchmark has always been and raises questions about what prior self-consistency results were actually measuring.

This connects directly to the April 21 paper on unsupervised confidence calibration ('Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generation'), which also treats self-consistency as a signal worth distilling, but for a different purpose: estimating model confidence rather than boosting accuracy. Together, the two papers suggest self-consistency is doing something more general than majority-vote correction on symbolic steps — it may be functioning as a soft ensemble that smooths over retrieval noise as much as reasoning errors. That reframing matters because calibration and accuracy are related but distinct goals, and conflating the mechanism risks building confidence estimators on shaky theoretical ground.

If follow-up work replicates the recall gains on MedMCQA's harder clinical reasoning subset (where knowledge and inference are tightly coupled), that would support the generalization claim. If gains collapse there, the effect is probably specific to MMLU's particular knowledge-retrieval structure rather than a property of self-consistency itself.