Towards Intrinsic Interpretability of Large Language Models:A Survey of Design Principles and Architectures

The meaningful distinction here is between interpretability as an afterthought and interpretability as a structural constraint baked into training objectives and architecture choices from the start. Surveys of this kind matter less for their novelty than for codifying a design vocabulary that practitioners and regulators can actually use to compare approaches.

This lands in the middle of a busy week for interpretability research on Modelwire. The AtManRL paper from April 17 is a direct neighbor: it proposes making chain-of-thought reasoning faithful to actual model decisions through differentiable attention masks, which is precisely the kind of intrinsic mechanism this survey would categorize and evaluate. The ORCA framework covered April 16 takes the opposite approach, adding structural interpretability post-training to SVMs rather than designing it in, which illustrates the core tension the survey addresses. The Prototype-Grounded Concept Models piece from April 17 adds a third data point: even in vision models, bolting on interpretability after the fact produces concept misalignment that requires additional correction machinery. Taken together, the week's coverage makes a reasonable empirical case for why the intrinsic-versus-post-hoc question is genuinely open.

The survey's practical value will depend on whether any of the five architectural approaches it catalogs get adopted in a model that ships with verifiable interpretability guarantees before a major regulatory deadline, such as the EU AI Act's high-risk system requirements. If none of the frameworks appear in production deployments within 18 months, the taxonomy risks becoming a citation fixture rather than a design guide.