Code models encode type information across languages, interpretability study finds

TypeProbe doesn't just show that code models learn types; it reveals that type structure emerges as a learnable abstraction even from untyped source code, suggesting models extract semantic invariants rather than memorize surface patterns. This matters because it implies type reasoning is fundamental to how these models reason about program behavior.

This connects directly to the 'Knowing-Using Gap' work from earlier this week, which found that memorized knowledge often fails to integrate with downstream reasoning circuits. TypeProbe suggests the inverse: that semantic structure (types) does route through hidden states in a way that transfers across languages, implying type information is encoded in a form that generalizes. If types embed as a unified abstraction across Java and Python, that's evidence some learned representations avoid the isolation problem that plagues fine-tuned facts. This also bears on the Token-Flow Firewall concern about semantic corruption propagating through internal token flows; if type structure is recoverable and stable in hidden states, it becomes a potential anchor point for detecting when semantic integrity has been compromised.

If researchers can show that adversarial code perturbations that preserve surface syntax but corrupt type semantics cause measurable degradation in TypeProbe's recovered representations, that would confirm types are genuinely learned abstractions rather than artifacts of the probe itself. Watch for follow-up work applying this technique to detect type-level attacks on code models used in security-critical contexts.