Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors

Source

arXiv

arxiv.org/abs/2608.06300v1

Read the source

Source

Plain-Language Brief

Researchers analyze the fairness and interpretability of two automatic speaking assessment systems using Concept Activation Vectors (CAVs). They find that the recoverability and sensitivity to certain concepts depend on the representation and architecture of the model.

What the Document Says

  • The study focuses on automatic L2 speaking assessment systems in high-stakes settings.
  • The goal is to ensure scores are based on speaking proficiency, not irrelevant attributes like L1 or age.
  • Transformer-based foundation models improve the accuracy of these systems, but their black-box nature makes fairness analysis harder.
  • CAVs, which can detect bias towards unwanted attributes in feature-based graders, are extended for two neural speaking assessment systems: a text-based BERT grader and a multimodal Whisper grader.
  • CAVs represent human-interpretable concepts as directions in the model's activation space, allowing distinction between encoded concepts and their impact on predicted scores.
  • A gradient-based sensitivity metric quantifies concept influence.
  • The study shows that concept recoverability depends more on the probed representation and architecture than on the concept itself.
  • Sensitivity to concepts also varies by architecture.
  • Sparse Autoencoders (SAEs) make concepts more linearly recoverable, but reduce original activation-space sensitivity, particularly in low-dimensional layers.

Key Concepts Mentioned

  • L2 Speaking Assessment Systems
  • Bias Analysis
  • Concept Activation Vectors (CAVs)
  • Transformer-based foundation models
  • BERT
  • Whisper
  • Sparse Autoencoders (SAEs)

Claims Made by the Document

  • Automatic speaking assessment systems need to be fair and interpretable.
  • CAVs can detect bias in feature-based graders and are extended for neural speaking assessment systems.
  • Concept recoverability depends on the representation and architecture of the model, not just the concept itself.
  • Sensitivity to concepts varies by architecture.

Evidence Used

  • The study's focus on automatic L2 speaking assessment systems in high-stakes settings implies a concern for fairness and interpretability.
  • The use of CAVs for bias analysis and their extension for the studied neural speaking assessment systems suggests their applicability for detecting bias.

Methods / Approach

  • CAVs for bias detection
  • Gradient-based sensitivity metric
  • Sparse Autoencoders (SAEs)

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Automatic speaking assessment systems are increasingly deployed in high-stakes settings to mark second language (L2) learners' speaking tests, making it critical to show that their scores depend on speaking proficiency rather than irrelevant speaker attributes such as first language (L1) or age. Transformer-based foundation models have improved the accuracy of these L2 speaking graders, but their black-box representations make fairness and interpretability analysis more difficult. Building on prior work that used Concept Activation Vectors (CAVs) to detect bias towards unwanted attributes (`concepts') in feature-based graders, we extend CAV-based analysis to two neural speaking assessment systems: a text-based BERT grader and a speech-and-text multimodal grader based on Whisper. CAVs represent human-interpretable concepts as directions in a model's activation space, allowing us to distinguish between whether a concept is encoded in a model's internal representations and whether it influences the predicted score, the latter quantified using a gradient-based sensitivity metric. Since CAVs rely on linear separability, which is less likely in complex neural embedding spaces, we also investigate whether sparse autoencoders (SAEs) provide cleaner concept directions by learning CAVs in a sparse latent space and mapping them back to activation space. Our analysis shows that concept recoverability depends strongly on the representation and architecture being probed, rather than on the concept alone. Sensitivity to concepts is also architecture-dependent. SAEs make concepts more linearly recoverable, but attenuate the original activation-space sensitivity, especially in low-dimensional layers. These findings highlight the need to distinguish concept recoverability from concept influence when auditing bias in speaking assessment systems.