PatRe: A Full-Stage Office Action and Rebuttal Generation Benchmark for Patent Examination

PatRe reframes patent examination as inherently adversarial and iterative rather than a one-shot classification problem. The benchmark captures the full back-and-forth cycle where examiners raise objections and applicants must rebut with domain-specific legal and technical justification, exposing how LLMs fail at sustained argumentation under constraints, not just at single-turn reasoning.

This connects directly to two recent findings. The diagnostic study on procedural execution (May 1) showed that LLMs collapse from 61% to 20% accuracy as task steps increase from short to 95-step sequences, losing track of intermediate state rather than making reasoning errors. PatRe surfaces a similar fragility in a legal domain: models must maintain coherent position across multiple turns while respecting prior examiner feedback and technical precedent. Separately, RunAgent (May 1) tackled multi-step workflow execution by adding constraint-based validation and explicit control flow. PatRe's oracle and retrieval-augmented evaluation modes serve a similar function: they surface where models break down in structured, multi-turn contexts rather than hiding failures in aggregate scores.

If PatRe-trained models show measurable improvement on the rebuttal generation task but still fail to maintain consistency across turns 3 and beyond, that confirms the gap is not reasoning ability but procedural faithfulness under domain constraints. Watch whether patent offices or IP vendors adopt PatRe for model validation within the next 12 months; adoption would signal the benchmark has moved from academic exercise to operational relevance.