IUQ: Interrogative Uncertainty Quantification for Long-Form Large Language Model Generation

The real gap IUQ addresses is that most uncertainty quantification methods were designed for discrete outputs like classification labels or multiple-choice answers, where correctness is well-defined. Long-form generation breaks those assumptions because a response can be partially correct, internally inconsistent, or faithful to the prompt while still being wrong about the world.

This lands in the middle of a cluster of uncertainty-focused research published on the same day. The MADE benchmark for medical adverse event classification and the SegWithU paper for medical image segmentation both tackle uncertainty quantification in high-stakes domains, but neither confronts the free-form text problem IUQ targets. More directly relevant is the 'Diagnosing LLM Judge Reliability' paper, which found that aggregate consistency scores for LLM evaluators can look healthy at 96% while hiding per-document logical failures in a third to two-thirds of cases. IUQ's within-sample faithfulness metrics are essentially trying to catch exactly that kind of hidden inconsistency before it reaches an evaluator.

The meaningful test is whether IUQ's consistency metrics correlate with human preference judgments on established long-form benchmarks like ELI5 or ASQA. If that correlation holds at scale, the framework has practical value for evaluation pipelines; if it doesn't, it remains a theoretical instrument without a clear deployment path.