Adaptive multi-fidelity optimization with fast learning rates

The headline contribution is not just speed but adaptivity: most prior multi-fidelity methods require the practitioner to specify how closely the cheap approximation tracks the true objective, a parameter that is rarely known in practice. Kometo removes that requirement while still achieving near-optimal rates, which is the part that matters for real deployment.

This connects most directly to the optimizer benchmarking work covered yesterday ('Benchmarking Optimizers for MLPs in Tabular Deep Learning'), which found that practitioners are often using suboptimal defaults simply because better alternatives impose configuration burdens. Kometo addresses a structurally similar problem one level up: the cost of tuning the optimization process itself, not just the optimizer within it. Beyond that specific link, recent coverage here has clustered around making expensive ML operations cheaper through smarter approximation, a thread that also runs through AdaSplash-2's histogram-based attention work from the same day. Kometo fits that pattern but targets a different bottleneck, the outer loop of hyperparameter and architecture search rather than the inner loop of training.

The practical test is whether Kometo holds its convergence advantage on neural architecture search benchmarks like NAS-Bench-201, where fidelity mismatch between proxy and full training is well-documented and measurable. If independent replications on those benchmarks confirm the rates, the 'no prior knowledge' claim is credible; if results degrade there, the theory may not survive contact with realistic approximation gaps.