Can "AI" Be a Doctor? A Study of Empathy, Readability, and Alignment in Clinical LLMs

The buried finding is directional: the problem isn't that these models lack medical knowledge, it's that their default output register is wrong for the setting. Complexity and negative sentiment aren't bugs introduced by fine-tuning; they appear to be baseline tendencies of general-purpose models that healthcare deployments inherit without correction.

This connects directly to the reliability problems surfaced in our April 16 coverage of LLM judge evaluation. That paper found logical inconsistencies in roughly one-third to two-thirds of pairwise comparisons even when aggregate scores looked healthy. The clinical LLM study is a downstream version of the same problem: aggregate capability metrics can look acceptable while per-instance failures are clinically significant. The DiscoTrace paper from the same week adds another layer, showing that LLMs systematically favor breadth over selectivity in how they construct responses, which maps neatly onto why physician-authored notes and model-generated notes diverge in tone and complexity even when the factual content overlaps.

Watch whether GPT-5 or Claude release healthcare-specific system prompt guidance or fine-tuned variants within the next two quarters. If they do, this paper will likely be cited as part of the justification; if neither acts, that signals the labs view alignment for clinical communication as the deployer's problem, not theirs.