Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs

Source

arXiv

arxiv.org/abs/2606.26987v1

Read the source

Source

Plain-Language Brief

Researchers have found that two open-source language models, Apertus-8B-Instruct-2509 and Gemma-4-E4B-it, have internal representations (emotion vectors) that can encode emotions such as valence and arousal. The valence representation for both models shows a correlation with human psychological structure, but there are differences in how these representations emerge across the depth of the models.

What the Document Says

  • The research identifies emotion vectors in two open-source language models: Apertus-8B-Instruct-2509 and Gemma-4-E4B-it.
  • Emotion contrast vectors were extracted across all layers of the models, using two model-generated corpora.
  • Valence geometry was recovered for both models, with correlation coefficients $r = 0.76$ and $0.83$ for Apertus and Gemma respectively, which is close to the $r = 0.81$ reported for Claude Sonnet 4.5.
  • There are differences in how valence representations emerge across model depth for the two models. In Gemma-4-E4B-it, valence is strongly encoded in early layers but collapses towards later layers, while Apertus-8B-Instruct-2509 exhibits the opposite pattern.
  • Arousal encoding is sensitive to the extraction corpus. Both models show stronger PC2--arousal alignment with Gemma-generated stories than Apertus-generated ones.
  • The study open-sources their experiment code and dataset for further investigation of emotion representations across language model architectures.

Key Concepts Mentioned

  • emotion vectors
  • valence
  • arousal
  • language models
  • open-source
  • corpora

Claims Made by the Document

  • Emotion vectors have been identified in two open-source language models.
  • Valence representations for both models show a correlation with human psychological structure.
  • There are differences in how valence representations emerge across model depth for the two models.
  • Arousal encoding is sensitive to the extraction corpus.

Evidence Used

  • Recent work identified emotion vectors in Claude Sonnet 4.5, which are internal representations that encode emotion concepts, causally influence behavior, and exhibit geometry mirroring human psychological structure. We test the generality of these findings in two open-weight models, Apertus-8B-Instruct-2509 and Gemma-4-E4B-it.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Recent work identified emotion vectors in Claude Sonnet 4.5, which are internal representations that encode emotion concepts, causally influence behavior, and exhibit geometry mirroring human psychological structure. We test the generality of these findings in two open-weight models, Apertus-8B-Instruct-2509 and Gemma-4-E4B-it, extracting emotion contrast vectors across all layers, using two model-generated corpora. We recover valence geometry for both models, with peak PC1--valence correlations of $r = 0.76$ and $r = 0.83$, approaching the $r = 0.81$ reported for Claude.Beyond replication, we observe notable differences in how valence representations emerge across model depth. In Gemma-4-E4B-it, valence is strongly encoded in early layers but collapses towards later layers, whereas Apertus-8B-Instruct-2509 exhibits the opposite pattern, with valence representations absent in early layers, but emerging at mid depths. Arousal encoding, in contrast, is sensitive to the extraction corpus: both models show stronger PC2--arousal alignment with Gemma-generated stories ($r$ up to $0.45$) than Apertus-generated ones ($r \leq 0.21$), suggesting arousal-relevant cues are unevenly distributed across generated corpora. We open-source our experiment code and dataset for reproducible investigation of emotion representations across language model architectures.