Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Bias Analysis of L2 Speaking Assessment Systems Using Concept Activation Vectors

Source

arXiv

arxiv.org/abs/2608.06300v1

Read the source

Source

Plain-Language Brief

Researchers used Concept Activation Vectors (CAVs) to analyze the bias of two automatic L2 speaking assessment systems: a text-based BERT grader and a multimodal grader based on Whisper. The study shows that the recoverability and sensitivity of concepts depend on the representation and architecture of the model being analyzed, and sparse autoencoders (SAEs) can make concepts more linearly recoverable but may attenuate original activation-space sensitivity.

What the Document Says

  • Researchers performed a bias analysis using CAVs on two neural speaking assessment systems: a text-based BERT grader and a multimodal grader based on Whisper.
  • The study shows that the recoverability and sensitivity of concepts depend on the representation and architecture of the model being analyzed.
  • SAEs can make concepts more linearly recoverable but may attenuate original activation-space sensitivity.

Key Concepts Mentioned

  • Concept Activation Vectors (CAVs)
  • automatic speaking assessment systems
  • second language (L2) learners' speaking tests
  • Transformer-based foundation models
  • bias
  • first language (L1)
  • age
  • text-based BERT grader
  • speech-and-text multimodal grader
  • Whisper
  • sparse autoencoders (SAEs)

Claims Made by the Document

  • The recoverability and sensitivity of concepts depend on the representation and architecture of the model being analyzed.
  • SAEs can make concepts more linearly recoverable but may attenuate original activation-space sensitivity.

Evidence Used

  • The text mentions that CAVs are used to detect bias towards unwanted attributes in feature-based graders and then extended to analyze two neural speaking assessment systems. The study finds that concept recoverability depends on the representation and architecture of the model, and SAEs can make concepts more linearly recoverable but may attenuate original activation-space sensitivity.

Methods / Approach

  • Using Concept Activation Vectors (CAVs) to analyze the internal representations and predicted scores of two neural speaking assessment systems.
  • Investigating whether sparse autoencoders (SAEs) provide cleaner concept directions by learning CAVs in a sparse latent space and mapping them back to activation space.

Limitations Stated by the Authors

  • The text mentions that since CAVs rely on linear separability, which is less likely in complex neural embedding spaces, the study investigates whether SAEs can provide cleaner concept directions.

Limitations Noted by the Agent

Agent interpretation — not from the source.

  • The study only analyzes two specific neural speaking assessment systems. It's unknown whether the findings generalize to other types of L2 speaking assessment systems.
  • The study does not discuss the potential impact of cultural or regional factors on bias in L2 speaking assessment.

Open Questions

  • What are the implications of these findings for improving the fairness and interpretability of automatic L2 speaking graders?
  • How can the limitations identified by the authors be addressed to further improve the CAV-based bias analysis method?

Notes for Future Q&A

May include interpretation. Labeled where it does.

  • The study focuses on showing that the scores produced by automatic speaking assessment systems depend on speaking proficiency rather than irrelevant speaker attributes, but it does not discuss the overall accuracy or effectiveness of these systems.

Raw Abstract / Excerpt

Automatic speaking assessment systems are increasingly deployed in high-stakes settings to mark second language (L2) learners' speaking tests, making it critical to show that their scores depend on speaking proficiency rather than irrelevant speaker attributes such as first language (L1) or age. Transformer-based foundation models have improved the accuracy of these L2 speaking graders, but their black-box representations make fairness and interpretability analysis more difficult. Building on prior work that used Concept Activation Vectors (CAVs) to detect bias towards unwanted attributes (`concepts') in feature-based graders, we extend CAV-based analysis to two neural speaking assessment systems: a text-based BERT grader and a speech-and-text multimodal grader based on Whisper. CAVs represent human-interpretable concepts as directions in a model's activation space, allowing us to distinguish between whether a concept is encoded in a model's internal representations and whether it influences the predicted score, the latter quantified using a gradient-based sensitivity metric. Since CAVs rely on linear separability, which is less likely in complex neural embedding spaces, we also investigate whether sparse autoencoders (SAEs) provide cleaner concept directions by learning CAVs in a sparse latent space and mapping them back to activation space. Our analysis shows that concept recoverability depends strongly on the representation and architecture being probed, rather than on the concept alone. Sensitivity to concepts is also architecture-dependent. SAEs make concepts more linearly recoverable, but attenuate the original activation-space sensitivity, especially in low-dimensional layers. These findings highlight the need to distinguish concept recoverability from concept influence when auditing bias in speaking assessment systems.