Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space

The paper's use of story comprehension as a probe is doing real methodological work here: narratives force sequential belief updating in a way that isolated factual queries don't, making the geometric trajectory claim testable rather than post-hoc.

This connects most directly to 'Geometric Factual Recall in Transformers' from the same day, which found that transformers encode relational knowledge through geometric superposition rather than direct associative lookup. Together, these two papers are building a consistent picture: transformer internals are more structured and lower-dimensional than the parameter count implies, whether the task is static fact retrieval or dynamic belief updating across a narrative. The ORCE paper on confidence calibration is also adjacent, since if belief trajectories are predictable and structured, that has direct implications for when and why verbalized confidence goes wrong. The connection to ORBIT's catastrophic forgetting work is weaker but worth noting: if adaptation follows constrained geometric paths, forgetting may be partly a story about trajectories leaving recoverable manifold regions.

The key test is whether the low-dimensional manifold structure holds under distribution shift, specifically on inputs where models are known to fail. If researchers apply this framework to the failure cases identified in wildfire or domain-shifted settings and the trajectories become irregular or high-dimensional before errors occur, that would make this a practical diagnostic rather than a descriptive one.