STEB: Style Text Embedding Benchmark

Source

Source name: arXiv
Source kind: paper
Authors / organization: Rafael Rivera Soto, Anna Wegmann, Cristina Aggazzotti
Venue: arXiv: cs.CL
Published: 2026-06-30
Collected: 2026-07-01
URL: http://arxiv.org/abs/2606.31741v1
PDF: https://arxiv.org/pdf/2606.31741v1
DOI: UNKNOWN

Plain-Language Brief

The authors introduce a new comprehensive benchmark called Style Text Embedding Benchmark (STEB), designed to evaluate style embeddings, which have been previously assessed in a fragmented manner. The STEB includes 96 datasets across 7 languages and various applications, such as authorship verification, AI-text detection, etc. The authors find that semantic embeddings struggle with stylistic tasks and no single style embedding excels across all tasks.

What the Document Says

The text presents a new benchmark called Style Text Embedding Benchmark (STEB).
STEB is intended to standardize the evaluation of style embeddings.
STEB consists of 96 datasets across 7 languages.
The applications covered by STEB include authorship verification, AI-text detection, and others.

Key Concepts Mentioned

Style Text Embedding Benchmark (STEB)
Semantic embeddings
Style embeddings
Authorship verification
AI-text detection

Claims Made by the Document

Semantic embeddings fail in stylistic tasks.
There is no style embedding that excels across all tasks.

Evidence Used

The authors report the findings from evaluating style embeddings using STEB.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

While semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented, with each work relying on their own set of tasks and datasets. To bridge this gap, we introduce the Style Text Embedding Benchmark, a comprehensive open-source benchmark intended to standardize the evaluation of style embeddings. STEB encompasses 96 datasets across 7 languages, spanning applications such as authorship verification, authorship retrieval, AI-text detection, probing of linguistic features, and others. We find that semantic embeddings consistently fail in stylistic tasks, and that there is no style embedding that is universally superior across all tasks evaluated. We open-source the STEB code base at: https://github.com/rrivera1849/STEB.