Vision-language models hide uncertainty despite internal awareness

The key insight is architectural: VLMs possess the internal signal to recognize insufficient evidence but lack the training or design to act on it. This isn't a perception problem masquerading as one, which reframes where fixes should target.

This finding sits directly alongside the August LLM uncertainty work, particularly the Gricean retreat paper, which identified the same encode-but-fail-to-express pattern in text models. TRAPSBench adds a vision-specific stress test and a calibration metric (PECS) that penalizes confident wrong answers, whereas prior VLM benchmarks like SciFigBench focused on whether models admit uncertainty when visual input is corrupted or absent. The convergence across modalities (text and vision) in the past week suggests the field is converging on a shared diagnosis: models have the mechanistic ingredients but coordination failures prevent deployment of those ingredients. This is distinct from the instruction-tuning work, which showed tuning increases verbalized confidence independent of accuracy; here the problem is the opposite (internal knowledge, external silence).

If researchers successfully patch this via training interventions (e.g., auxiliary losses that reward abstention when internal uncertainty is high) and the fix generalizes to held-out video domains beyond TRAPSBench within the next six months, that confirms the bottleneck is truly expression rather than perception. If the fix fails to transfer or requires model-specific tuning, the diagnosis was incomplete.