LLM accuracy collapses as candidate sets grow, study finds

The paper isolates a specific failure mode: not that LLMs can't find the right answer in a large set, but that they lose the ability to rank it confidently above plausible alternatives. This is distinct from long-context degradation and suggests the problem is architectural rather than informational.

This connects directly to 'The Decomposition Tax' study from late September, which found that LLMs hemorrhage accuracy when intermediate pipeline stages lose problem context. Both papers identify a common thread: LLMs degrade not because they lack information, but because they lose the structural coherence needed to maintain decision confidence across stages or scale. Where decomposition taxes information visibility at boundaries, choice overload taxes confidence separation at scale. Together they suggest the real bottleneck in production systems isn't retrieval or reasoning capacity, but the model's ability to preserve signal-to-noise ratios under complexity.

If the same models tested here show recovery when given explicit ranking instructions or confidence calibration prompts, that confirms the issue is remediable through inference design. If accuracy remains flat regardless of prompting strategy, watch whether practitioners begin building external ranking layers (learned rerankers or symbolic filters) as a standard post-hoc step rather than relying on end-to-end model selection.