What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search

The study's most actionable finding is buried in the framing: 'incremental refinement' and 'semantic localization' are measurable behavioral signatures, not vague intuitions, which means they could eventually serve as selection criteria when choosing which model to deploy for search-based tasks rather than defaulting to the highest benchmark scorer.

This connects directly to the shortest-path generalization paper from April 16, which found that LLMs fail not on initial problem setup but when tasks require sustained, recursive progress over longer horizons. That paper diagnosed a ceiling on systematic problem-solving; this new work essentially explains part of why that ceiling exists: models that can't refine incrementally collapse under extended search. The logical reasoning faithfulness study from April 21 adds a complementary angle — models prefer to report failure rather than produce a wrong answer, which suggests the 'good optimizer' behavior identified here may be rarer than benchmark scores imply.

Watch whether follow-up work tests these trajectory signatures against reinforcement-trained models specifically, since the EVPO paper from April 21 suggests critic quality during post-training directly shapes the kind of iterative signal a model learns to trust. If trajectory quality correlates with explained-variance metrics from that training regime, the two research threads converge into something practically useful for model selection.