LEX-EC: A Lexical Evidence-Channel Audit Framework for Zero-Shot LLM Personality Classification in Black-Box Settings

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Brittany Harbison, Ashok K. Goel
• Venue: arXiv: cs.CL
• Published: 2026-07-27
• Collected: 2026-07-28
• URL: http://arxiv.org/abs/2607.24435v1
• PDF: https://arxiv.org/pdf/2607.24435v1
• DOI: UNKNOWN

Plain-Language Brief

Researchers introduce LEX-EC, a framework for analyzing the interpretability of large language models in assigning personality labels from text. The framework identifies and separates marginal distribution effects from trait-associated signals using prevalence diagnostics, agreement diagnostics, and lexical ablation.

What the Document Says

• A new audit framework called LEX-EC is introduced for analyzing the interpretability of large language models in assigning personality labels from text.
• The framework combines prevalence and agreement diagnostics with controlled lexical ablation to distinguish marginal distribution effects from trait-associated signals.
• The study demonstrates that various text genres have different profiles for these associations, with free-form essays containing the broadest but still weak signal.
• In graduate student introductions, an observable Extraversion association weakened after masking.
• Single Facebook statuses yield little stable evidence, indicating a possible lower bound of content or length.
• Masking topical and demographic content weakened some associations while leaving others detectable from function words, affective terms, and cognitive-style vocabulary.
• Linguistic prompting shifted model self-explanations but did not eliminate topical content.
• The framework evaluates classification prevalence, item-level association, chance-corrected agreement, persistence under lexical restriction, and prompt sensitivity in model-generated explanations.
• Across datasets, models, and prompts, LEX-EC characterizes how trait associations may vary with available lexical evidence.

Key Concepts Mentioned

• Large language models
• Personality labels
• Interpretability
• LEX-EC framework
• Prevalence diagnostics
• Agreement diagnostics
• Lexical ablation
• Text genres
• Free-form essays
• Graduate student introductions
• Facebook statuses
• Function words
• Affective terms
• Cognitive-style vocabulary
• Linguistic prompting

Claims Made by the Document

• Various text genres have different profiles for personality associations.
• Graduate student introductions exhibit an observable Extraversion association that weakens after masking.

Evidence Used

• Demonstration of differences in profiles across various text genres.
• Observation of a weakening Extraversion association in graduate student introductions after masking

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Large language models may easily assign personality labels from text, but model interpretability remains an open problem. To address this gap, we introduce LEX-EC, a reusable black-box audit framework combining prevalence and agreement diagnostics with controlled lexical ablation to distinguish marginal-distribution effects from trait-associated signal recoverable under restricted evidence. Using this framework, we illustrate how various text genres may exhibit sharply different profiles: free-form essay text contains the broadest, but still weak, signal; in graduate student introductions, an observable Extraversion association weakened after masking; and single Facebook statuses yield little stable evidence even in a trait-balanced sample, indicating a possible lower bound of content or length. Masking topical and demographic content weakened some associations while leaving others detectable from function words, affective terms, and cognitive-style vocabulary. Linguistic prompting shifted model self-explanations but did not eliminate topical content. LEX-EC jointly evaluates classification prevalence, item-level association, chance-corrected agreement, persistence under lexical restriction, and prompt sensitivity in model-generated explanations. Across datasets, models, and prompts, LEX-EC characterizes how trait associations may vary with available lexical evidence, introducing a novel application of lexical methods to black-box interpretability in personality labeling.