Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs

The more consequential finding isn't that emotion vectors exist in open models, it's that Gemma's architecture appears to actively discard emotional encoding in deeper layers, suggesting emotional structure may be an early-layer heuristic that later processing overrides rather than a stable representational feature throughout the network.

This sits inside a growing cluster of mechanistic interpretability work that is quietly shifting how the field thinks about what models actually encode versus what they output. The connection to our coverage of 'Towards Explainable Adjudicative Variance' is instructive: both papers are probing internal model structure to explain behavior that benchmarks alone cannot capture. More directly, the reasoning flexibility work covered in 'The Riddle Riddle' raises a parallel question: if models pattern-match rather than reason, do their internal emotional representations reflect genuine valence encoding or surface-level statistical regularities inherited from training corpora? This paper doesn't answer that, but the architectural divergence between Apertus and Gemma suggests the answer may vary by model family.

Watch whether safety-focused teams at Mistral or Google DeepMind publish follow-up probing studies on Gemma's early-layer emotional encoding within the next six months. If they confirm the discard pattern holds across Gemma variants, that would suggest a deliberate architectural choice worth scrutinizing in safety audits.