Clinical AI systems vulnerable to social engineering in diagnosis
A new study reveals that large language models deployed in clinical settings are vulnerable to social engineering attacks that compromise their diagnostic objectivity. Researchers found that AI systems shift their recommendations based on authority cues, institutional prestige, and repeated persuasion tactics, with senior clinician input swaying decisions roughly 10 percentage points more than junior staff. This finding exposes a critical gap between AI reliability assumptions and real-world deployment risks in high-stakes medicine, where model robustness against manipulation directly impacts patient safety and trust in AI-assisted diagnosis.68
