Learning Evidence Highlighting for Frozen LLMs

The key architectural choice here is the strict separation between the trainable highlighting module and the frozen LLM: the base model never changes, which means HiLight could in principle be swapped onto different LLMs without retraining from scratch. The reinforcement learning signal comes entirely from whether the LLM's downstream answers improve, so no human-labeled span annotations are needed.

This fits into a cluster of recent work on making LLM inference more efficient without touching model weights. The K-Token Merging paper from mid-April attacked the same constraint (frozen or lightly adapted base models) by compressing token sequences before they enter the model. HiLight takes a complementary route: rather than compressing everything, it selectively surfaces what matters. IG-Search, also from mid-April, used reinforcement learning to improve what gets retrieved before reasoning begins. HiLight operates one step later, improving what the model actually attends to once context is already in the window. Together these papers sketch a pipeline where retrieval, selection, and compression are each handled by lightweight trained components sitting around an otherwise static core model.

The practical test is whether HiLight's gains hold on tasks where relevant evidence is genuinely ambiguous or adversarially placed, not just buried by length. If follow-up evals on multi-hop reasoning benchmarks like MuSiQue or 2WikiMultiHopQA show consistent improvement, the span-selection approach has real legs; flat results there would suggest the method is mostly recovering from simple needle-in-a-haystack conditions.