OpenAI researchers want to predict how often AI models will fail before launch

The harder problem buried here is not prediction itself but calibration: a forecast of failure rates is only useful if the model of what counts as failure matches how users actually experience harm or error in production, which is notoriously difficult to define consistently across deployments.

This is largely disconnected from recent activity in our archive, as we have no prior coverage to anchor it to. It belongs to a broader thread in AI safety research around the gap between benchmark performance and deployment reliability, a problem that has surfaced repeatedly in academic literature and in post-mortems from labs after high-profile model misbehavior. The core tension is that benchmarks are static and curated, while production traffic is adversarial, diverse, and shifts over time. Forecasting failure rates before launch requires a model of that distribution, which is precisely what labs lack before users interact with the system at scale.

Watch whether OpenAI publishes a formal methodology or evaluation framework within the next six months that other labs can apply to their own models. If the work stays internal and unpublished, it functions as a proprietary deployment tool rather than a contribution to shared safety infrastructure.