Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Detecting LLM-Generated Tokens in Human--LLM Coauthored Text

Source

arXiv

arxiv.org/abs/2607.21458v1

Read the source

Source

Plain-Language Brief

A new method for detecting parts of text generated by a language model (LLM) within human-written text is introduced. This method operates at the token level, reducing variability in adjacent tokens while using an adaptive Lepski-type rule to select the bandwidth based on local authorship structure.

What the Document Says

  • The paper introduces a new method for detecting parts of text generated by a language model (LLM) within human-written text.
  • The method operates at the token level and builds on existing token-level detection scores.
  • The key idea is to smooth adjacent token scores to reduce variability, using an adaptive Lepski-type rule for bandwidth selection based on local authorship structure.
  • The method is simple to implement and does not require token-level labeled data for training.
  • Theoretically, the paper characterizes this trade-off and shows that the proposed method achieves favorable mean square error performance in estimating the underlying signal.
  • Empirically, the method demonstrates strong performance against a wide range of baselines in both synthetic datasets and a realistic dataset.
  • A publicly accessible website implementing the methods is deployed.

Key Concepts Mentioned

  • human-AI collaborative writing
  • language model (LLM)
  • token-level detection
  • Lepski-type rule

Claims Made by the Document

  • The proposed method can detect parts of text generated by LLMs within human-written text.
  • The method performs favorably in estimating the underlying signal.
  • The method demonstrates strong performance against a wide range of baselines.

Evidence Used

  • The paper introduces and describes the new method in detail.
  • Theoretical analysis and empirical evaluation are provided to support the claims.

Methods / Approach

  • smoothing adjacent token scores
  • adaptive Lepski-type rule for bandwidth selection

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

The method may not perform optimally for certain types of text or in specific contexts.

Open Questions

  • How well does the method perform on real-world documents with varying authorship styles and language models?
  • Can the method be improved further to achieve even better performance?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The paper focuses on detecting parts of text generated by a language model, not the entire document. It also assumes that the underlying language model is known.

Raw Abstract / Excerpt

The rise of human-AI collaborative writing has created a growing need for fine-grained detection methods that support localizing likely LLM-generated content in mixed-authorship documents. Existing methods for detecting LLM-generated text mainly focus on document-level classification and cannot identify which parts of the text are generated by LLMs. This paper introduces a new method to address this urgent need. Our method operates at the token level, the natural unit of modern language models, and builds on existing token-level detection scores. The key idea is to smooth adjacent token scores to reduce their variability, while using an adaptive Lepski-type rule to select the bandwidth according to the local authorship structure. Our method is simple to implement and does not require token-level labeled data for training. Theoretically, we characterize this trade-off and show that the proposed method achieves favorable mean square error performance in estimating the underlying signal. Empirically, we demonstrate strong performance of our method against a wide range of baselines in both synthetic datasets and a realistic dataset. We deploy a publicly accessible website that implements the methods as well.