LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning

The interesting move here is borrowing the intuition from model quantization, where high-magnitude weights carry disproportionate signal, and applying it to the training process itself rather than to inference compression. That reframing is what makes LongAct structurally different from prior long-context RL work, which typically attacks the problem through context window extension or positional encoding tricks.

This sits in productive tension with the K-Token Merging paper covered the same day, which also targets computational overhead in long sequences but from the inference side via latent-space compression. Together they sketch two complementary pressure points on the same bottleneck: training efficiency and serving efficiency. The IG-Search paper from the same batch is also relevant, since it applies step-level RL rewards to improve reasoning over retrieved context, a problem that gets harder as context length grows. LongAct's sparse optimization framing could, in principle, make that kind of fine-grained RL more tractable at scale, though the papers don't reference each other.

The key test is whether LongAct's activation-sparsity approach holds up on standard long-context benchmarks like RULER or LongBench at context lengths above 128k tokens. If third-party reproductions show degraded gains past that threshold, the quantization analogy may not transfer cleanly to the long-tail of positional distributions.