FIELD NOTES TOPIC
attention mechanisms
3 notes, newest first.
Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference
A research paper titled 'Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference' proposes a new method for creating small language models. Instead of adapting large models to CPUs, the authors…
HeadCast: Casting Attention Heads for Efficient Autoregressive Video Generation
Researchers propose HeadCast, a framework that improves the efficiency of autoregressive video diffusion models without requiring re-training. The framework categorizes attention heads into four types based on their…
HAS: Highlight-guided Attention Steering for Multimodal LLM Video Summarization
In this research paper, the authors propose a method called Highlight-guided Attention Steering (HAS) for improving video summarization using Multimodal Large Language Models (M-LLMs). HAS aims to enhance the…
