ReLeVAnT: Relevance Lexical Vectors for Accurate Legal Text Classification

The interesting design choice here is the deliberate rejection of LLMs as a component, not just as a baseline. ReLeVAnT is built around the premise that legal classification workflows have latency and cost constraints that make even lightweight LLM calls impractical at scale, which reframes the contribution as an infrastructure argument rather than a pure accuracy claim.

This sits in productive tension with recent coverage of LLM evaluation reliability. The 'Diagnosing LLM Judge Reliability' paper from mid-April found that one-third to two-thirds of documents show logical inconsistencies when LLMs perform pairwise comparisons, which is precisely the kind of failure mode that makes a deterministic, lexical approach attractive for high-stakes legal contexts. If LLM judges are unreliable at the document level, substituting them with n-gram scoring in a binary classification pipeline is a defensible engineering choice, not a step backward. That said, ReLeVAnT's court-filing focus is fairly narrow, and most of the recent coverage here has centered on LLM agent behavior and inference efficiency rather than legal NLP specifically.

The real test is whether the contrastive scoring holds up across different legal jurisdictions and document types beyond the paper's evaluation set. If an independent replication on federal versus state court filings shows significant accuracy degradation, the method's generalizability claim weakens considerably.