CoRe: A Continuously Reward-Finetuned LLM Query Rewriter for Multi-Stage Context-Aware Relevance in Web-Scale Video Search
A major short-video search platform deployed CoRe, a continuously retraining LLM query rewriter that closes the gap between offline reward modeling and production ranker behavior. The system uses the live multimodal relevance model as its reward signal and employs a cost-efficient semi-online preference optimization loop to support weekly redeployment across millions of instances. This represents a practical solution to a core production challenge: keeping LLM components aligned with downstream systems as data distribution shifts, without incurring prohibitive retraining costs. The five-month deployment validates that reward-aligned continuous finetuning is operationally feasible at scale.
Modelwire context
Analyst takeThe paper's most underreported detail is the weekly redeployment cadence across millions of instances, which implies the platform has already solved the MLOps scaffolding that most teams treat as the hard part. The reward signal being the live multimodal ranker itself is a meaningful architectural choice: it means the query rewriter is optimizing for a moving target by design, which is either robust or fragile depending on how stable that ranker actually is.
CoRe sits in a cluster of work this week about closing the gap between how LLMs are trained and how they behave inside larger production pipelines. The implicit reasoning recommendation paper ("Implicit Reasoning for Large Language Model-based Generative Recommendation") identified nearly the same structural problem in a different domain: LLM components trained in isolation misalign with the downstream system they serve. CoRe's semi-online preference loop is essentially a production answer to that failure mode, applied to search rather than recommendation. CacheRL's cached-rollout approach addresses a related cost problem in agent training, suggesting that compute-efficient continuous alignment is becoming a shared constraint across multiple deployment contexts.
Watch whether the unnamed platform publishes ranker stability metrics alongside future CoRe iterations. If the live multimodal reward model shifts significantly between retraining cycles and query rewriter quality holds, that validates the feedback loop design. If quality degrades during ranker transitions, the architecture has a hidden brittleness the paper does not address.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsCoRe · DPO · Mixed Preference Optimization · LLM query rewriter
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.