
OpenAI's Sol model shows 38% on custom tests, 7.8% on standard ARC-AGI-3
OpenAI's latest frontier model, GPT-5.6 Sol, reveals a widening gap between lab conditions and real-world benchmarking. The model achieves 38.3 percent on ARC-AGI-3 using OpenAI's proprietary test harness with extended reasoning, but drops to 7.8 percent in the official environment, where Anthropic's Opus 5 scores 30.2 percent without such scaffolding. This discrepancy signals a critical tension in how frontier labs measure progress: custom infrastructure can inflate apparent capability gains, while standardized benchmarks remain the only reliable cross-vendor comparison. For practitioners and investors, the gap underscores that claimed breakthroughs require scrutiny of test conditions before informing deployment decisions.73















