Self-Policy Distillation via Capability-Selective Subspace Projection

The key distinction the summary gestures at but doesn't fully unpack is the 'subspace projection' mechanism: SPD doesn't just filter outputs by quality score, it isolates the geometric region of the model's representation space associated with a specific capability, then distills only from outputs that activate that region. That's a structural claim about where skill lives in a model, not just a data curation heuristic.

This connects most directly to the data and training methodology thread running through recent coverage. The piece on 'Understanding Data Temporality Impact on Large Language Models Pre-training' raised a similar underlying question: what structure in training signal actually shapes what a model learns, versus what is noise or artifact? SPD is essentially asking the same question at the fine-tuning stage. Both papers push back against the assumption that more signal, applied broadly, is always better. The connection to tokenization or political consistency work is thinner and not worth forcing.

The real test is whether capability-selective filtering holds up when the target capability is something genuinely hard to operationalize as a subspace, like multi-step reasoning rather than a narrow stylistic dimension. If follow-up work applies SPD to coding or math benchmarks with held-out test sets and shows transfer beyond the filtered domain, the structural claim is credible. If results stay narrow, this is a sophisticated filtering trick, not a general self-improvement framework.