LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning

Researchers propose LongAct, a reinforcement learning technique that leverages high-magnitude activation patterns in query and key vectors to improve long-context reasoning in LLMs. The method treats long-context RL as a sparse optimization problem, drawing parallels to model quantization to identify which weights matter most for training efficiency.

MentionsLongAct

Modelwire summarizes — we don’t republish. The full article lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.