RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models

The real contribution here is not LoRA itself, which is well-established, but the claim that hidden-state geometry carries a readable signal about which layers are doing meaningful representational work. Using a path-simplification algorithm originally designed for cartographic line reduction to read that signal is an unusual methodological choice that deserves scrutiny on its own terms.

This connects directly to the K-Token Merging paper from April 16, which also used LoRA-adapted models as the backbone for a compression technique. That paper treated LoRA as a given infrastructure choice; RDP LoRA steps back and asks whether the standard practice of uniform or heuristic layer selection is actually sound. Together they sketch a broader pattern: the field is now optimizing around LoRA rather than replacing it, treating it as a substrate worth refining rather than a placeholder for something better. Neither paper challenges the core adapter paradigm, which means the compounding assumption that LoRA-adapted layers generalize well across tasks remains largely unexamined in both works.

The key test is whether RDP-selected layer configurations transfer across model families: if the geometric signal identifies consistent layer patterns on Mistral and Llama architectures independently, the method has structural validity; if the selected layers shift substantially between architectures, it may be fitting to training data geometry rather than capturing something fundamental about depth and representation.