FIELD NOTES TOPIC
language modeling
3 notes, newest first.
The State-Prediction Separation Hypothesis
Researchers propose a new model for language modeling called 'state-prediction separation' which aims to improve efficiency by separating the prediction of the next token and the storage of useful state for future…
When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
The study examines the use of multiple language models (LLMs) to improve accuracy over single models and finds that their performance is limited by a factor called 'beta'. Beta represents the rate at which all models…
SPIRAL: Learning to Search and Aggregate
Researchers introduced SPIRAL, a framework for improving language model reasoning at test time. The framework uses three primitives - sequential reasoning within a trace, independently sampled parallel traces, and…
