Report the Floor: A Training-Free Conformal Interval Is a Mandatory Baseline for Probabilistic Time-Series Forecasting
A new study challenges the forecasting community's reliance on learned models by demonstrating that an untrained conformal baseline, combining last-value prediction with split-conformal quantiles, outperforms sophisticated neural approaches across 2,217 real-world time series. The finding exposes a methodological gap in probabilistic forecasting benchmarks, where weak or absent baselines have masked incremental gains from complex models. This matters for practitioners building production forecasters and signals that the field may be optimizing for publication rather than practical utility.
Modelwire context
ExplainerThe paper's sharpest contribution isn't the baseline itself but the audit it implies: if 2,217 series can't reliably distinguish a last-value conformal interval from a trained neural model, then the benchmark infrastructure the field has been using to declare progress is structurally compromised.
This connects directly to the calibration paper covered the same day, 'Investigating Calibration Challenges in Probabilistic Electricity Price Forecasting,' which found that learned models systematically sacrifice reliability for sharpness. Both papers are diagnosing the same underlying problem from different angles: the scoring rules and evaluation setups rewarding complexity are not measuring what practitioners actually need. The STGCN depth study from the same batch adds a third data point, showing that architectural elaboration in traffic forecasting also fails to justify its overhead. Taken together, these three papers suggest a reproducibility pressure building across time-series subfields, where benchmark design has been quietly subsidizing model complexity.
Watch whether the Monash or LOTSA benchmark maintainers formally incorporate a conformal baseline into their evaluation suites within the next two conference cycles. If they do, reported gains from new probabilistic forecasters will compress noticeably, confirming the gap was real.
This analysis is generated by Modelwire’s editorial layer from our archive and the summary above. It is not a substitute for the original reporting. How we write it.
MentionsMonash · LOTSA · METR-LA · NPTS · Conformal Seas
Modelwire Editorial
This synthesis and analysis was prepared by the Modelwire editorial team. We use advanced language models to read, ground, and connect the day’s most significant AI developments, providing original strategic context that helps practitioners and leaders stay ahead of the frontier.
Modelwire summarizes, we don’t republish. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.