Continuous Batching vs. Static Batching in LLM Serving
Continuous batching keeps GPU slots filled while static methods leave them idle.
Naomi Ibrahim
Columnist
Naomi Ibrahim
Naomi Ibrahim is a columnist at Inference Storage Review covering inference throughput. Based in Barcelona, Naomi has written for Inference Storage Review since 2015.
1 story · Barcelona