FIELD NOTES TOPIC
ai inference
2 notes, newest first.
Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets
The authors present a method to distribute Large Language Model (LLM) inference across multiple Intel AI PCs using pre-compiled pipeline shards. This allows for efficient use of idle AI PCs with integrated GPUs and…
HCRMap: Pressure-Aware Hot-Expert Residency Mapping for 3.5D MoE Chiplet Inference
The paper proposes HCRMap, a framework for managing hot expert replicas in 3.5D Mixture-of-Experts (MoE) inference systems. The aim is to dynamically place and reuse hot experts across different memory tiers to…
