Pretraining Data Can Be Poisoned through Computational Propaganda

Source

• Source name: arXiv
• Source kind: paper
• Authors / organization: Victoria Graf, Hannaneh Hajishirzi, Noah A. Smith, David Kohlbrenner, Kyle Lo
• Venue: arXiv: cs.AI
• Published: 2026-07-16
• Collected: 2026-07-17
• URL: http://arxiv.org/abs/2607.15267v1
• PDF: https://arxiv.org/pdf/2607.15267v1
• DOI: UNKNOWN

Plain-Language Brief

The research discusses the potential for malicious content to be injected into large-scale pretraining datasets used for training language models (LMs). The authors propose a novel analysis method, HalfLife, to estimate the inclusion of adversarial content in such data. They suggest that open discussion interfaces can be used as a vector for such attacks.

What the Document Says

• Poisoning pretraining data can result in harmful behaviors in LMs.
• Existing work on poisoning pretraining data focuses on established data sources but ignores interactions with data curation pipelines.
• The study demonstrates that poisoning attacks are possible beyond this limited setting using a web-scale content injection mechanism: public discussion interfaces.
• HalfLife is introduced as an analysis for estimating adversarial content inclusion in LM training data.
• The authors use HalfLife to explore the feasibility of poisoning pretraining corpora at web scale through open discussion interfaces.
• The research suggests that third-party webpage content could be a vector for attacking language model pretraining.

Key Concepts Mentioned

• pretraining data
• language models (LMs)
• poisoning attacks
• public discussion interfaces
• HalfLife

Claims Made by the Document

• Poisoning pretraining data can lead to harmful behaviors in LMs.
• Third-party webpage content could be a vector for attacking language model pretraining.

Evidence Used

• The study demonstrates the potential for malicious content to be included in pretraining data and the feasibility of using public discussion interfaces for such attacks.

Methods / Approach

• Introduction of HalfLife analysis for estimating adversarial content inclusion in LM training data.

Limitations Stated by the Authors

The study does not provide specific examples or case studies of poisoning attacks on pretraining data.

Limitations Noted by the Agent

Agent interpretation — not from the source.

The research does not investigate methods to detect and mitigate the impact of such poisoned data on language models.

Open Questions

• How can poisoned data be detected and mitigated in language model training?

Notes for Future Q&A

May include interpretation. Labeled where it does.

The study highlights the need for further research into detecting and mitigating the effects of poisoned pretraining data on language models.

Raw Abstract / Excerpt

Poisoning pretraining data can introduce harmful behaviors to LMs that are difficult to detect and mitigate. Prior work on poisoning pretraining data has largely exploited established data sources such as Wikipedia, which do not represent the large scale and heterogeneity typical of pretraining corpora, and has ignored the interaction between poisoned data and data curation pipelines. We demonstrate that poisoning attacks on pretraining data are feasible beyond this limited setting through an existing web-scale content injection mechanism: public discussion interfaces. Additionally, to measure whether malicious content is included after web crawling and data curation, we introduce HalfLife, a novel analysis for estimating adversarial content inclusion in web-crawl based LM training data. We use HalfLife to explore the feasibility of poisoning pretraining corpora at web scale through open discussion interfaces. Our analysis demonstrates the importance of estimating whether poison injections are included in pretraining data, and establishes third-party webpage content as a possible vector for attacking language model pretraining.