
Upstage scales Solar Open to 250B parameters with 1M-token context
Upstage has scaled Solar Open to 250 billion parameters using a mixture-of-experts architecture designed for long-context agent reasoning. The model achieves a 1M-token context window through a hybrid attention mechanism combining softmax and linear layers, enabling entire agent trajectories to fit in memory. By initializing from Solar Open 1 and leveraging higher-quality training data, the team maintained efficiency under fixed compute budgets. This represents a significant step toward production-grade models capable of sustained multi-step reasoning, directly addressing a key bottleneck in agentic AI systems.62




























