The Distillation Game: Adaptive Attacks & Efficient Defenses

The sharpest contribution here is not the defense itself but the formalization of the adaptive attacker, an adversary who observes the defense and adjusts queries accordingly. Most prior work benchmarks defenses against static attack strategies, which is roughly equivalent to testing a lock only against people who don't know it exists.

This is largely disconnected from recent activity in our archive, as we have no prior coverage of model extraction, distillation attacks, or API security research to anchor it to. It belongs to a cluster of work sitting at the intersection of model security and inference-time intervention, a space that has grown quietly alongside the commercial API economy. The practical stakes are highest for any organization monetizing a proprietary model through an API, where output quality and theft resistance are now in direct tension. That tension is not new as a concern, but formalizing it as a strategic game with measurable equilibria is a meaningful step toward making it tractable for deployment teams.

Watch whether major inference providers (Fireworks, Together, or the hyperscalers) reference or adopt Product-of-Experts style output perturbation in their API documentation within the next six months. Adoption there would confirm the defense is practical at production latency; silence would suggest the overhead cost is prohibitive outside research settings.