Adnan KhanAdnanProduct & AI
San Francisco Bay Area
Get in touch

Source note3 min read

Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference

Source

arXiv

arxiv.org/abs/2608.20210v1

Read the source

Source

  • Source name: arXiv
  • Source kind: paper
  • Authors / organization: Christos Koutsiaris
  • Venue: arXiv: cs.IR
  • Published: 2026-08-20
  • Collected: 2026-08-21
  • URL: http://arxiv.org/abs/2608.20210v1
  • PDF: https://arxiv.org/pdf/2608.20210v1
  • DOI: UNKNOWN
  • Registry group: academic_apis_and_preprints
  • Evidence role: canonical_academic_index
  • Language: en
  • Country / region: UNKNOWN

Plain-Language Brief

A research paper titled 'Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference' proposes a new method for creating small language models. Instead of adapting large models to CPUs, the authors designed an architecture specifically for one user and one token at a time, using 4-bit weights on ordinary CPUs. The model was trained from scratch on 59.9B tokens and outperforms other models in terms of speed, quality metric, and file size on a five-task benchmark.

What the Document Says

  • The paper presents a new architecture for small language models specifically designed to run on CPUs.
  • The model uses 4-bit weights and has 18 blocks, of which only 6 use full attention.
  • The other 12 blocks use short convolutions whose memory is two timesteps wide.
  • The model was trained from scratch on 59.9B tokens and scores higher than several other models on a five-task benchmark.
  • The model outperforms GPT-2 124M, Pythia-160M, OPT-125M, GPT-neo-125M, and MobileLLM-125M in terms of speed, quality metric, and file size.
  • The model uses less data than these other models but still performs better.
  • The model's validation bits-per-byte is 0.8685.
  • The authors compared the hybrid architecture with a conventional all-attention model of the same size on the same data.
  • The hybrid model won in the chosen quality metric, performed equally well on downstream tasks, produced a smaller file, and decoded faster at 2048 tokens of context.
  • The speed advantage increases with the length of the context.
  • A simple bandwidth calculation predicts only 1.17x speed increase, suggesting that memory volume alone does not explain the gap.
  • The authors also report what did not work: an unmitigated 4-bit quality cost, roughly half the convolution channels ending up inert and impossible to remove, and a vocabulary larger than the model size warrants.

Key Concepts Mentioned

  • language models
  • CPU
  • convolution-attention hybrid
  • 4-bit weights
  • 59.9B tokens
  • five-task benchmark

Claims Made by the Document

  • The proposed architecture is designed for CPUs.
  • The model outperforms other models in terms of speed, quality metric, and file size.
  • The model uses less data but still performs better than other models.
  • A simple bandwidth calculation does not fully explain the speed advantage.

Evidence Used

  • The paper describes a new architecture designed for CPUs.
  • The paper reports that the model outperforms other models in various metrics.
  • The paper mentions that the model uses less data than other models but still performs better.
  • The paper explains that a simple bandwidth calculation predicts only 1.17x speed increase.

Methods / Approach

UNKNOWN

Limitations Stated by the Authors

UNKNOWN

Limitations Noted by the Agent

Agent interpretation — not from the source.

UNKNOWN

Open Questions

UNKNOWN

Notes for Future Q&A

May include interpretation. Labeled where it does.

UNKNOWN

Raw Abstract / Excerpt

Small language models are usually built like large ones and then squeezed onto a CPU afterwards. We did the opposite: we fixed the target first, one user, one token at a time, 4-bit weights, ordinary CPU, and chose the architecture to suit it. The result keeps full attention in only 6 of its 18 blocks. The other 12 use short convolutions whose memory is two timesteps wide no matter how long the conversation gets, so two thirds of the network never re-reads a growing cache. Trained from scratch on 59.9B tokens, the model scores 47.31 on a five-task benchmark against a bar of 42.20 that was fixed before training began. It beats GPT-2 124M, Pythia-160M, OPT-125M and GPT-neo-125M, all trained on three to six times more data, and exceeds MobileLLM-125M's published score despite that model seeing a trillion tokens. Validation bits-per-byte is 0.8685. To check the architecture rather than the training recipe, we trained a conventional all-attention model of the same size on the same data, and wrote down the winning condition before scoring either. The hybrid won the chosen quality metric by 0.81%, matched it on downstream tasks, produced a 6.3% smaller 4-bit file, and decoded 1.76x faster at 2048 tokens of context, 2.08x against an external model of similar size. In every measurement the speed advantage is near zero at an empty context and grows with length, which is what the mechanism predicts and what a merely leaner model would not show. A simple bandwidth calculation predicts only 1.17x, so memory volume alone does not explain the gap. We also report what did not work: an unmitigated 4-bit quality cost, roughly half the convolution channels ending up inert and impossible to remove, and a vocabulary larger than this model size warrants.