FIELD NOTES TOPIC
distributed computing
2 notes, newest first.
Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets
The authors present a method to split large machine learning models into smaller parts called pipeline shards, which can then be run on multiple Integrated AI Processors (AIPCs) over an ordinary network. They show that…
Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets
The authors propose a method to utilize multiple Intel AI PCs with integrated GPUs and NPUs, even when they have insufficient memory for large models like a 70B-parameter LLM. The method involves breaking the model…
