MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval

The retrieval dimension is the part worth pausing on: MathNet doesn't just test whether a model can solve problems, it tests whether embedding systems can detect mathematical equivalence across different notations and languages, which is a distinct and underexplored failure mode for RAG pipelines used in educational or scientific tools.

The benchmark wave we've been tracking since mid-April continues here. CoopEval (covered April 16) tested LLM agents on behavioral tasks; QuantCode-Bench tested code generation under domain constraints; MADE tested classification under label uncertainty. What those share with MathNet is a push toward benchmarks that stress-test a specific, narrow capability rather than general performance, which is a methodological correction to the broad leaderboard culture. The related coverage doesn't surface a direct competitor to MathNet specifically, but the pattern is consistent: researchers are building evaluation infrastructure faster than models are being evaluated on it.

Watch whether any of the major embedding model providers (Cohere, OpenAI, Voyage) publish retrieval scores on MathNet's equivalence-detection split within the next two quarters. If they don't, that silence is itself informative about how seriously the field takes math-specific retrieval as a benchmark axis.