Large-scale safety audit finds chatbot failures shift with teen responses

The study's core contribution isn't just testing more scenarios, but systematically measuring how chatbot responses diverge based on conversational context and prompt engineering. Most existing safety benchmarks treat each interaction as isolated; this work reveals that small shifts in how a teen phrases a follow-up question or how the system is instructed can flip a safe response into an unsafe one.

This connects directly to two recent findings. The PersonaChatBench work from late October showed that varying user profiles exposes performance gaps invisible in single-profile testing. SAFE-Teen extends that insight into the temporal domain: it's not just different users, but the same user across multiple turns where prior context shapes downstream safety outcomes. The temporal reasoning gap identified in the TASC benchmark from today is also relevant here, since mental health conversations inherently unfold over time and require models to track safety constraints across exchanges. Together, these papers signal that the field is moving beyond point-in-time safety checks toward interaction-level evaluation.

If any of the 18 chatbots tested here (OpenAI, Anthropic, Google, etc.) announce new safety controls specifically for mental health conversations in the next 60 days, that signals the vendors are treating these findings as actionable. Alternatively, watch whether regulatory bodies cite SAFE-Teen in upcoming guidance on teen-facing AI products; that would indicate the benchmark has crossed from research artifact into policy relevance.