Upstage scales Solar Open to 250B parameters with 1M-token context
Source published ·Modelwire updated
Original coverage: arXiv cs.CL ↗·How Modelwire adds context
The development
Upstage has scaled Solar Open to 250 billion parameters using a mixture-of-experts architecture designed for long-context agent reasoning. The model achieves a 1M-token context window through a hybrid attention mechanism combining softmax and linear layers, enabling entire agent trajectories to fit in memory. By initializing from Solar Open 1 and leveraging higher-quality training data, the team maintained efficiency under fixed compute budgets. This represents a significant step toward production-grade models capable of sustained multi-step reasoning, directly addressing a key bottleneck in agentic AI systems.
Modelwire’s AI-generated summary of coverage from arXiv cs.CL.
Modelwire analysis
Analyst takeOur AI-generated reading of the wider context and the next developments to watch.
The 1M-token context window is the headline, but the more consequential detail is the architectural choice to combine softmax and linear attention layers rather than adopt pure linear attention, a tradeoff that preserves quality on short contexts while extending range, and one that carries real inference cost implications Upstage has not yet publicly benchmarked at production throughput.
The attention efficiency problem Solar Open 2 is navigating sits directly alongside what ELSAA addressed earlier this week: the quadratic cost of attention at long sequence lengths is the binding constraint, and hybrid approximation is emerging as the practical answer rather than any single clean solution. Meanwhile, the MoE scaling choice puts Upstage in the same infrastructure territory as the SLAI T-Rex work on DeepSeek-V4, where trillion-parameter MoE training economics are being stress-tested across different hardware stacks. What remains unaddressed is the safety surface that OpenSkillRisk flagged: a 250B-parameter model with 1M-token context designed explicitly for agentic trajectories is precisely the deployment profile where third-party skill risks compound.
Watch whether Upstage publishes inference throughput numbers at full 1M-token context on commodity hardware within the next 60 days. If they do not, the context window claim is a training-time capability, not a deployable one.
This interpretation is generated from the summary above and available source metadata. Our methodology · Report an error
MentionsUpstage · Solar Open 2 · Solar Open 1 · Mixture-of-Experts
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.CL originally reported this story as “Solar Open 2 Technical Report”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.