Polish clinical AI outpaces physicians and frontier LLMs on primary care diagnostics
A Polish clinical AI system called Doctorina substantially outperformed both physicians and frontier language models on primary-care diagnostics across 150 synthetic cases, achieving 82% diagnostic concordance versus 57% for doctors and 85% for leading LLMs on differential reasoning. The study signals that specialized medical AI trained on adaptive information-gathering workflows may create a meaningful capability gap over general-purpose models in constrained clinical domains, raising questions about deployment readiness and the role of domain-specific fine-tuning in regulated healthcare settings.
Modelwire context
Skeptical readThe study doesn't disclose whether Doctorina was trained on or tuned against these specific diagnostic cases, or how the synthetic cases were generated. Without that detail, the 82% figure could reflect benchmark overfitting rather than genuine clinical superiority.
This connects directly to the sycophancy and robustness findings from early September. The 'Measuring LLM Sycophancy' work showed that frontier models collapse under sustained pressure in multi-turn settings, yet this study tests LLMs in a constrained, single-turn diagnostic format where they face no adversarial pushback. A system that performs well on a static benchmark may still fail when physicians or patients challenge its reasoning iteratively. The real question isn't whether Doctorina wins on 150 curated cases, but whether it maintains that edge when users apply the kind of sustained disagreement that destabilizes current LLMs.
If Doctorina's authors release the synthetic case generation pipeline and confirm the model was not trained on these specific cases, the claim gains credibility. If they don't, or if independent teams reproduce the benchmark with held-out case distributions, watch whether the performance gap shrinks below 5 percentage points. That would suggest the benchmark is narrow rather than indicative of real-world diagnostic advantage.
Coverage we drew on
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsDoctorina · Kimi K3 · Claude Opus 5 · Claude Opus
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Performance of Clinical AI System and Physicians and Frontier Language Models in primary care diagnostics”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.