FIELD NOTES TOPIC
key-value store
1 note, newest first.
SMetric: Rethink LLM Scheduling for Serving Agents with Balanced Session-centric Scheduling
The paper proposes a new method for scheduling Language Model (LLM) requests called SMetric, which aims to improve the performance of serving agents in terms of tokens per second (TPS). The current methods prioritize…
