Shubhra Pandit

vLLM Release Roundup: What’s New in vLLM v0.6.4?

2:4 Sparse Llama: Smaller Models for Efficient GPU Inference

We Ran Over Half a Million Evaluations on Quantized LLMs: Here's What We Found