ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning

The core architectural bet here is that transformer layers are not independent enough to warrant independent weight perturbations. ShadowPEFT treats cross-layer parameter sharing as a feature rather than a constraint, which is a different philosophical starting point than most LoRA variants that compete on rank selection or initialization.

This lands in a busy week for PEFT research. RDP LoRA, covered the same day, attacks a related problem from the opposite direction: instead of redesigning how updates are structured, it uses hidden-state geometry to decide which layers should receive LoRA updates at all. Together the two papers suggest the field is converging on the view that treating all transformer layers as equivalent candidates for adaptation is wasteful, even if they disagree on the remedy. The looped transformer stability work from April 16 is also loosely relevant, since fixed-point behavior across repeated layer passes is conceptually adjacent to what a shared shadow module must handle when it evolves across depth.

The meaningful test is whether ShadowPEFT's shared module holds up on tasks requiring diverse, layer-specific representations, such as multi-hop reasoning benchmarks, where per-layer independence in LoRA might actually be load-bearing. If ablations on those tasks appear in a follow-up or replication within the next two months, that will clarify whether centralization is a genuine efficiency gain or a capacity trade-off.