RASP-Tuner: Retrieval-Augmented Soft Prompts for Context-Aware Black-Box Optimization in Non-Stationary Environments

The key constraint RASP-Tuner targets is the black-box condition: the optimizer cannot inspect gradients or model internals, only query outputs. That restriction is common in real production deployments where the model is a third-party API or a compiled artifact, and it makes most standard fine-tuning approaches inapplicable from the start.

The retrieval component here shares conceptual ground with IG-Search (covered April 16), which also uses retrieval to condition model behavior at inference time rather than retraining from scratch. Both papers are working on a similar intuition: that a well-structured retrieval step can substitute for expensive reoptimization. Beyond that specific parallel, RASP-Tuner sits closer to the optimizer benchmarking work from the same week ('Benchmarking Optimizers for MLPs in Tabular Deep Learning'), which flagged that practitioners are still searching for practical alternatives to standard update rules when default assumptions break down. RASP-Tuner is essentially addressing the same practitioner pain from a different angle.

The real test is whether the retrieval corpus scales gracefully: if the method degrades when the archive of past regimes grows large or contains noisy entries, the practical deployment story weakens considerably. Watch for follow-up ablations on retrieval corpus size and contamination in any extended version or workshop submission.