LLMs show under 6% agreement with expert physics literature searches

The paper doesn't clarify whether the 6% overlap reflects genuine retrieval gaps or misalignment between how the study prompted models versus how domain experts naturally search. If researchers typically use LLMs to expand or cross-check curated lists rather than bootstrap searches from scratch, this benchmark may measure the wrong task.

This connects directly to the WorkSurface-Bench finding from late July, which identified that enterprise agents struggle with knowledge source routing before retrieval even begins. Both studies reveal that current models lack the meta-cognitive layer to identify what kind of search strategy a task requires. The physics literature review problem is a domain-specific instance of the same routing failure: models don't know whether to prioritize citation networks, keyword matching, or temporal relevance the way expert physicists do implicitly.

If the researchers release ablation data showing performance when given explicit search strategy hints (e.g., 'prioritize recent papers citing X'), overlap should jump significantly. If it doesn't, that confirms the gap is semantic understanding, not just retrieval. Watch whether follow-up work tests whether fine-tuning on physics-specific search logs improves overlap above 15 percent within six months.