Rethinking XAI Evaluation: A Human-Centered Audit of Shapley Benchmarks in High-Stakes Settings
Source published ·Modelwire updated
Original coverage: arXiv cs.LG ↗·How Modelwire adds context
The development
Researchers tested eight competing Shapley value variants across fraud detection and risk workflows with 3,735 professional analyst reviews, finding that standard quantitative metrics for explainability don't correlate with what actually helps humans make decisions in high-stakes settings.
Modelwire’s AI-generated summary of coverage from arXiv cs.LG.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The deeper finding here isn't just that metrics are imperfect: it's that the entire benchmarking infrastructure for Shapley-based XAI may be optimizing for the wrong objective from the start, measuring mathematical properties of explanations rather than whether those explanations change analyst behavior in useful ways.
This connects most directly to the 'White-Box Signal-Subspace Probe' paper from the same day, which also challenges opaque learned representations by substituting interpretable signal components. Both papers are pushing toward a similar position: that interpretability tools need to be validated against what practitioners actually do with them, not just against internal consistency measures. The broader cluster of April 24 arXiv work on this site is otherwise focused on efficiency and control problems, so this paper and the WG-SRC piece form a small but coherent thread about the gap between what models expose and what humans can use.
Watch whether fraud detection vendors or financial regulators cite this study when revisiting model explainability requirements. If a regulatory body references human-centered XAI evaluation criteria in updated guidance within the next 12 months, this line of research will have found real institutional traction.
This interpretation is generated from the summary above and the archive coverage cited below. Our methodology · Report an error
Coverage behind this analysis
These archive entries ground the connection in our analysis. They are ordered by source publication date, with links to our coverage and the original sources.
·arXiv cs.LG
Operational Feature Fingerprints of Graph Datasets via a White-Box Signal-Subspace Probe
Researchers introduce WG-SRC, a white-box diagnostic tool that decodes what graph neural networks learn during node classification by decomposing message passing into interpretable signal components. The method replaces opaque learned representations with fixed graph-signal dictionaries, enabling practitioners to diagnose which mechanisms a dataset actually requires.
MentionsShapley values
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.