How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoning

The key buried detail is that this work targets the answer token's reading behavior specifically, not the reasoning trace generation itself. That's a narrower and less-studied intervention point: the model has already done its thinking, and the question is whether it's consulting that thinking faithfully when it writes the final answer.

This connects most directly to 'From Tokens to Steps: Verification-Aware Speculative Decoding' from mid-April, which also used internal model signals rather than external reward models to improve reasoning reliability. Both papers are working the same seam: the model's own activations contain quality signals that are currently being left on the table. The 'Generalization in LLM Problem Solving' piece on shortest-path tasks is also relevant context, since it showed models failing at longer reasoning horizons due to recursive instability — a failure mode that scattered attention during answer generation could help explain mechanistically. The DiscoTrace work on answering strategies is adjacent but focused on discourse structure rather than attention mechanics, so the overlap is loose.

The real test is whether the proposed steering method holds up on multi-step symbolic reasoning benchmarks beyond the quantitative tasks studied here — if it degrades on something like MATH Level 5 or formal proof tasks, the attention pattern finding may be domain-specific rather than a general reliability signal.