vLLM is an open-source library for large language model inference and serving. Released in 2023 by researchers at UC Berkeley, it introduced PagedAttention, a technique that manages the attention KV cache in blocks much like virtual memory pages in an operating system, reducing wasted GPU memory and substantially increasing throughput.
It supports a wide range of models published on Hugging Face and provides an OpenAI-compatible API server so that existing applications can use it directly. It also supports continuous batching, quantization, and distributed inference across multiple GPUs, and is one of the most widely used inference engines for self-hosting open-source LLMs.