Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note2 min read

Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias

Source

arXiv

arxiv.org/abs/2607.11871v1

Read the source

Source

Plain-Language Brief

The research investigates bias in Large Language Models (LLMs) acting as judges by exploring representation-level accounts, focusing on the hidden state of the LLM, complementing input-output level studies. The study presents three findings: (1) biased inputs are displaced along a low-dimensional subspace in the judge's hidden state, (2) manipulating this subspace affects scoring in both directions, and (3) a simple linear projection using bias-direction features anticipates judge failures on unseen benchmarks.

What the Document Says

  • Seven judges were used to examine bias types in LLMs.
  • Bias was found to have a representation-level account in the judge's hidden state.
  • Biased inputs occupy a tight activation manifold, while biased ones are displaced along a low-dimensional, type-specific subspace.
  • This subspace sharpens with depth and is recovered consistently by three families of estimators.
  • Steering hidden states along this subspace affects scoring in both directions.
  • Three findings were reported across seven bias types and nine benchmarks.
  • The study demonstrates causal control and operational prediction using the representation-level account.
  • A simple linear projection onto the same bias-direction features anticipates judge failures on unseen benchmarks.
  • The project's webpage is available at https://xzx34.github.io/unfair-judge/

Key Concepts Mentioned

  • LLM-as-judge
  • bias
  • hidden state
  • geometry
  • causal control
  • operational prediction

Claims Made by the Document

  • Bias in LLMs has a representation-level account.
  • Steering hidden states along a specific subspace affects scoring.

Evidence Used

  • Biased inputs occupy a tight activation manifold while biased ones are displaced along a low-dimensional, type-specific subspace.
  • Manipulating this subspace affects scoring in both directions.
  • A simple linear projection onto the same bias-direction features anticipates judge failures on unseen benchmarks.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Existing studies of LLM-as-judge scoring bias work predominantly at the input-output level: they perturb inputs, measure score deltas, and propose prompt-level mitigations. We argue that the same biases admit a representation-level account in the judge's hidden state, complementary to the input-output view and operationally useful in ways it does not afford. We report three findings, across seven judges, seven bias types, and nine benchmarks. Geometry: baseline judging inputs occupy a tight activation manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth and is recovered consistently by three families of estimators. Causal control: steering hidden states along this subspace drives scoring in both directions, forward shifts reproducing biased scoring on clean inputs and reverse shifts restoring baseline scoring on biased ones, while matched-norm random directions produce shifts an order of magnitude smaller. Operational: a simple linear projection onto the same bias-direction features anticipates judge failures on three entirely unseen benchmarks, substantially outperforming text-based alternatives. Reading bias as activation geometry, rather than as input-output noise, unifies geometric structure, causal control, and operational prediction within a single framework. The project page is available at https://xzx34.github.io/unfair-judge/