Convolutional audio encoders structurally block access to pitch and timbre features

The paper's core claim is not just that models fail to learn frequency primitives, but that the failure is structural and predictable given the architecture itself. This suggests the limitation isn't a training data or objective problem that more scale could fix, but rather a hard constraint baked into how convolutions downsample information.

This connects directly to the interpretability work from earlier this week on Structured Sparse Autoencoders and steering networks through constraints. Those papers assume we can identify what features networks should learn and then either extract or enforce them. This audio work suggests some features may be architecturally inaccessible regardless of how well we interpret the model afterward. The implication cuts deeper: if frequency representations are bottlenecked by design, then interpretability tools alone cannot tell us what the model could theoretically learn, only what it actually does learn within those constraints.

If researchers demonstrate that replacing strided convolutions with alternative downsampling methods (learned pooling, attention-based aggregation, or frequency-preserving transforms) recovers interpretable pitch/timbre representations without sacrificing audio quality, that confirms the bottleneck is architectural rather than fundamental to end-to-end learning. If quality remains unchanged but interpretability doesn't improve, the constraint is real.