Self-Guided Plan Extraction for Instruction-Following Tasks with Goal-Conditional Reinforcement Learning

The key architectural bet here is that plan extraction and reward signal are co-trained rather than sequenced: the model isn't handed a decomposition schema and then optimized against it, which is how most instruction-following pipelines work. That joint training loop is what the benchmark numbers are actually measuring, and readers should hold that distinction in mind before generalizing the results.

This connects directly to IG-Search, covered here on April 16, which also applies step-level reinforcement learning signals to improve LLM reasoning without relying on trajectory-level rewards. Both papers are working on the same underlying problem: how to give a language model a denser, more informative training signal than a single end-of-episode outcome. Where IG-Search anchors its reward in retrieved document quality, SuperIgor anchors it in plan coherence relative to a goal condition. The two approaches are complementary rather than competing, and together they suggest a broader shift toward reward shaping at intermediate reasoning steps rather than at final outputs.

The stochastic environment results are promising, but the real test is whether SuperIgor's joint training holds up on long-horizon benchmarks with sparse natural-language goals, such as those used in the shortest-path generalization study also covered April 16. If performance degrades at longer horizons the way that paper's LLMs did, the co-training loop may be smoothing over the same recursive instability rather than solving it.