Efficient document VLM matches human annotation costs in regulated workflows

The paper's actual contribution is narrower than the headline suggests: it's not that fine-tuning closes the gap universally, but that selective curation on hard examples lets a smaller open-source model match larger baselines on specific document tasks. The deployment constraint (single H100, no API calls) is the real driver, not a general principle.

This work sits directly between two recent findings. The Visual Insensitivity Gap paper showed that VLMs often ignore visual input entirely, raising questions about whether benchmark improvements reflect genuine multimodal reasoning. Meanwhile, the LLM-as-Judge mechanistic analysis revealed how evaluation systems actually assign quality scores. This paper sidesteps both problems by using difficulty-aware curation to build training signal, but it doesn't address whether the resulting model actually uses visual evidence or merely learns spurious correlations. The John Deere chatbot deployment pattern also applies here: domain-specific fine-tuning on proprietary data creates defensibility where API-dependent systems fail in regulated settings.

If the authors release ablations showing that visual regions remain predictive in their fine-tuned model (via attention or perturbation analysis similar to the Themis mechanistic work), that confirms the quality gains are real. If they don't, the cost parity claim may reflect better language modeling on document metadata rather than improved document understanding. Watch for follow-up work applying this curation strategy to other VLM tasks outside document classification.