Automated coreset selection simplifies LLM unlearning without pre-labeled data
Researchers propose GRACE, a gradient-guided method that automates the construction of forget and retain datasets for LLM unlearning without requiring pre-labeled corpora. The technique uses seed examples of undesired behavior to compute a forget direction, then selects compact coresets whose gradients align with that direction while preserving model utility through orthogonal projection. This addresses a critical gap in unlearning workflows: most prior work assumes clean, pre-specified datasets, but real-world requests often arrive as sparse behavioral examples. GRACE's data-selection automation could accelerate deployment of unlearning in production systems where manual curation is infeasible, making it relevant to compliance, safety, and model maintenance pipelines.
Modelwire context
ExplainerThe paper's actual contribution is narrower than it might appear: GRACE doesn't solve unlearning itself, but rather automates the data-selection bottleneck that precedes unlearning. The key insight is using orthogonal projection to prevent the forget direction from degrading model performance on retained tasks, which is a constraint problem, not a learning problem.
This sits alongside the 'Localizing Global Discrepancies' work from late August, which also tackles data curation through influence-style analysis. Where that paper identifies which samples cause statistical failures, GRACE identifies which samples encode undesired behavior patterns. Both papers reflect a broader shift in the archive toward automating dataset construction rather than assuming clean, pre-labeled corpora. The unlearning angle is distinct from recent pretraining efficiency work (the curvature-conditioned momentum paper) and from mistake detection in VLMs, but shares the same operational constraint: real systems rarely arrive with perfectly specified training signals.
If GRACE's coreset selection reduces the size of forget datasets by more than 50 percent compared to full-corpus baselines while maintaining comparable unlearning fidelity on standard benchmarks (TOFU, MUSE), that confirms the gradient-alignment heuristic actually captures behavioral patterns. If coreset sizes remain large or if orthogonal projection introduces measurable utility loss on held-out tasks, the method is solving a narrower problem than claimed.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsGRACE
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.LG originally reported this story as “GRACE:Gradient-guided Coreset Selection for LLM Unlearning”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.