RUBEN: Rule-Based Explanations for Retrieval-Augmented LLM Systems

RUBEN's actual contribution is narrower than the summary suggests: it automates rule extraction from RAG outputs after the fact, but doesn't address whether those rules are faithful to the retrieval process itself or merely post-hoc rationalizations of what the LLM decided to output.

This connects directly to the Neural ArchEHR-QA work from the same day, which also tackles high-stakes QA grounding through evidence traceability and validation chaining. Both papers assume that in regulated domains, you need to show your work. But where Neural's approach chains retrieval and grounding validation into the forward pass, RUBEN operates backward from the output. The BICR paper on visual ungroundedness is also relevant here: both identify that confidence metrics and post-hoc explanations can mask whether the model actually used the retrieved context or just pattern-matched on language. The key difference is RUBEN focuses on extracting rules that explain decisions, while BICR diagnoses whether grounding happened at all.

If RUBEN's extracted rules successfully predict failure modes on held-out adversarial prompts that the original safety training missed, that validates the approach. If the rules turn out to be unfalsifiable or circular (e.g., 'the model output X because the retrieval contained X'), the method is descriptive but not diagnostic. Watch for follow-up work testing whether rule fidelity correlates with actual safety improvements in production RAG deployments.