Two layers suffice for piecewise linear functions, settling depth conjecture
Source published ·Modelwire updated
Original coverage: arXiv cs.LG ↗·How Modelwire adds context
The development
Researchers have resolved a longstanding question about neural network depth by proving that two hidden layers suffice to represent any continuous piecewise linear function, a fundamental building block of ReLU networks. The Caterpillar Net construction demonstrates this result through an explicit, non-recursive algorithm that computes the maximum of N inputs using quadratic and exponential-width layers, with direct geometric grounding. This settles prior conjectures about logarithmic depth requirements and clarifies the expressiveness boundaries of shallow architectures, informing both theoretical understanding of network capacity and practical design choices for practitioners building efficient models.
Modelwire’s AI-generated summary of coverage from arXiv cs.LG.
Modelwire analysis
ExplainerOur AI-generated reading of the wider context and the next developments to watch.
The Caterpillar Net doesn't just prove two layers work in principle; it provides an explicit, constructive algorithm with geometric grounding. Prior work conjectured logarithmic depth was necessary. This shows the answer is actually constant depth, but at a concrete cost: quadratic width in the first layer and exponential width in the second.
This connects directly to the KAN critique from October 15th, which exposed how architectural claims often depend heavily on problem structure rather than inherent model advantages. The Caterpillar Net result is the inverse: it establishes a hard theoretical boundary (two layers suffice for any continuous piecewise linear function) that applies universally, not just to favorable cases. It also complements the Neural Expansion framework from October 16th, which explains how overparameterized networks generalize. Here we see the flip side: what's the minimum depth needed for expressiveness? The answer clarifies the depth-width tradeoff that practitioners face when choosing between shallow-wide and deep-narrow architectures.
If follow-up work shows the exponential-width requirement in Brunck et al.'s construction can be reduced to polynomial width while maintaining two layers, that would signal the result is tight and practically relevant. If it cannot be improved, practitioners should expect that shallow networks require prohibitive width, making the depth-width tradeoff more concrete for real model design.
This interpretation is generated from the summary above and available source metadata. Our methodology · Report an error
MentionsReLU networks · Caterpillar Net · Brunck et al. · continuous piecewise linear functions
How this coverage is produced
Modelwire uses AI to generate summaries and context from source headlines, snippets, and selected archive coverage. Automated checks do not verify every claim, and items are not routinely reviewed by a person before publication. Zacaria Solis operates the site. Read the linked source for the full evidence and report errors through our corrections process.
Modelwire summarizes, we don’t republish. arXiv cs.LG originally reported this story as “Two hidden layer suffice: the caterpillar net”. The full content lives on arxiv.org. If you’re a publisher and want a different summarization policy for your work, see our takedown page.