LLMs leak secrets across topic shifts despite context isolation
Researchers have exposed a critical vulnerability in long-context LLMs: sensitive information disclosed early in conversations remains extractable even after topic shifts and adversarial probing. The PrivDrift benchmark demonstrates that across three major models, dialogue-level secret leakage persists at 38.7% to 54.6% rates, revealing that context window extensions may amplify privacy risks rather than contain them. This finding challenges assumptions about information compartmentalization in persistent assistant deployments and signals that production systems require stronger isolation mechanisms between conversation phases.
Modelwire context
ExplainerThe paper's core insight is that information compartmentalization fails not because models forget early secrets, but because adversarial topic shifts create plausible retrieval pathways. This inverts the assumption that longer context windows improve safety by keeping sensitive data 'contained' within a conversation.
This connects directly to the JevOut finding from the same day: naturally phrased contextual additions can flip model behavior in ways that aren't obvious from the core task alone. Here, the mechanism is different (secret extraction vs. decision routing), but the underlying vulnerability is identical: models treat context as a unified reasoning space rather than compartmentalized phases. Both papers expose gaps between how systems behave in isolation versus under realistic multi-turn deployment. The audio modality work (VeriSpeak, phonetic features) and the conspiracy detection agent paper both grapple with similar context-dependence problems, though in different domains.
If PrivDrift's leakage rates hold when tested on production-scale models (Claude 3.5, GPT-4o) with conversation lengths exceeding 50K tokens, that confirms the vulnerability scales with context size. If major LLM providers ship explicit conversation-phase isolation (separate system prompts, token-level access controls) within the next six months, that's a direct response to this threat model.
Coverage we drew on
- JevOut: Natural Context Can Flip Decision Models · arXiv cs.CL
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsPrivDrift · LLMs
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversations”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.