Researchers pinpoint identity storage in multimodal models, enable targeted removal
Researchers have identified where multimodal LLMs store identity information and developed a method to remove it without the typical privacy tradeoffs. The core insight is that identity data concentrates in early-to-mid decoder MLPs, layers that can be surgically modified without degrading vision capabilities. This matters because existing unlearning approaches either require a retain set (reintroducing privacy risks) or damage model perception broadly. The new technique, PAVA, targets these specific pathways, enabling post-deployment removal of individuals' data from production systems. This advances the practical feasibility of machine unlearning at scale, a growing requirement as regulation tightens around data rights.
Modelwire context
ExplainerThe paper's real contribution is not just identifying identity storage locations, but proving that removing identity information doesn't require keeping a retain set of non-identity data. Prior unlearning methods faced a hard tradeoff: either use a retain set (which reintroduces privacy leakage) or degrade the model broadly. PAVA breaks that tradeoff by targeting only the specific MLP layers where identity concentrates.
This connects directly to the mechanistic interpretability infrastructure work covered in MURANO (late August). Both papers assume that understanding where information lives in model internals is the prerequisite for safe intervention. Where MURANO standardized the tooling pipeline for interpretability experiments, this work demonstrates a high-stakes application of that pipeline: surgical removal of sensitive attributes without collateral damage. The causal tracing methodology here is the kind of attribution technique MURANO aims to make more accessible across research teams.
If major model providers (Anthropic, Meta, OpenAI) announce adoption of PAVA-style unlearning in their data removal workflows within the next 12 months, that signals regulatory pressure is driving real deployment. If no production systems implement this by mid-2027, the work remains academically interesting but operationally inert, suggesting either technical barriers at scale or insufficient legal mandate.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
Mentionsmultimodal large language models · PAVA · causal tracing · Fisher overlap
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Models”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.