BackTrend: Evaluating Scientific Weak-Signal Prediction via Backward Reconstruction

Source facts

BackTrend is a retrospective benchmark that asks systems to recover early problem-space and solution-space precursors for mature AI/ML topics under temporal evidence constraints. It contains 25 mature topics and 66 human-validated weak signals reconstructed from 2019–2024 publication trajectories, and evaluates frontier LLMs, RAG systems, and agentic research systems.

Analysis

This benchmark is directly relevant to the vault's trend methodology because it tests whether systems can identify precursors rather than merely summarize currently popular topics. The reported low coverage suggests that retrieval and plausible narrative generation are not sufficient for reliable weak-signal discovery.

Uncertainty/Doubt

• This is an arXiv preprint with a stated EMNLP Findings reference; the benchmark construction, human validation, and system results were not independently reproduced here.