OpenTumorBoard benchmark tests LLMs on real multidisciplinary clinical consensus

The benchmark doesn't just test diagnostic accuracy in isolation. It measures whether models can track and synthesize conflicting expert opinions across a live discussion, then propose actionable consensus. That's a different evaluation problem than most clinical AI papers attempt.

This fits a pattern we've tracked across recent papers: domain-specific benchmarks that expose gaps in how frontier models are actually evaluated. The USAI-Quant benchmark for vision-language models on geospatial reasoning and the VCRE-Fib work on ultrasound grading both identified similar mismatches between what existing evals measure and what real deployment requires. OpenTumorBoard extends that logic to a collaborative reasoning task. The common thread is that raw capability scores miss context-dependent competencies that matter in practice.

If the 14 models tested here show consistent performance ranking across both the question-answering and full consensus-building tasks, the benchmark is measuring stable model properties. If rankings flip between the two task types, it signals that collaboration ability is genuinely orthogonal to individual accuracy, which would reshape how clinical deployment decisions should weight model selection criteria.