EndPrompt: Efficient Long-Context Extension via Terminal Anchoring

The core trick is that transformers learn positional relationships, not raw sequence content, so you can fool the model into practicing long-range attention by placing tokens at distant positional indices during short training runs. The compute savings come from avoiding the quadratic attention cost that scales with actual sequence length, not from any architectural change to the model itself.

This sits in a broader pattern of inference and training efficiency work that avoids touching model weights or adding external components. The SIRA paper covered the same day pursues a structurally similar philosophy: rather than bolting on external tooling to fix a known weakness (hallucination there, context length here), both methods exploit what the transformer already does internally. The connection is not superficial. Both papers are essentially arguing that the architecture contains latent capacity that practitioners are currently paying too much to access. EndPrompt extends that argument into the training regime itself, which SIRA does not address.

The real test is whether models trained with EndPrompt on short sequences match full-length fine-tuned baselines on retrieval benchmarks like RULER or LongBench at the 32k-plus range. If independent groups replicate that parity within the next few months, the compute efficiency claim holds; if the gap widens past 16k tokens, the positional simulation is leaking.