Willison compares GPT-5.6 Sol Ultra and Claude Fable 5 on game generation

Willison's test isolates how aggressive optimization modes in GPT-5.6 Sol Ultra degrade coherence on multi-step creative workflows, while Claude Fable 5 maintains consistency at the cost of speed. This is the first public comparison showing the specific failure mode of speed-optimized inference on tasks that demand sustained reasoning across code generation phases.

This builds directly on Baseten's August 3rd analysis of inference optimization as a competitive differentiator (reference [5]). That piece framed speed and cost efficiency as now rivaling raw capability; Willison's experiment proves the qualifier: optimization gains don't transfer uniformly across task types. The raccoon heist test also echoes Karpathy's vibe-test methodology (reference [3]), but inverts it to measure where models break under real constraints rather than where they excel. Together, these stories suggest the frontier is shifting from 'which model is smartest' to 'which model degrades most gracefully under your specific optimization regime.'

If Codex Desktop ships a low-optimization mode variant within the next two months that recovers coherence on the same task, that signals OpenAI is treating inference strategy as a product lever rather than a fixed implementation detail. Conversely, if Claude Fable 5 remains the default for creative multi-step tasks across Anthropic's product line through Q4 2026, that confirms Anthropic is willing to trade throughput for consistency as a positioning choice.