SGLang is an open-source framework for fast serving of large language models and vision-language models. It was released in 2024 by LMSYS researchers centered at UC Berkeley and is distributed under the Apache 2.0 license.
Its key features include RadixAttention, which manages the KV cache of shared prompt prefixes in a radix tree and reuses it automatically across requests; fast generation of structured output constrained by regular expressions or JSON schemas; and a frontend language for expressing model calls and control flow in Python. Alongside vLLM, it is one of the most widely used LLM inference engines, and it also supports disaggregated prefill and decode serving and expert parallelism for large MoE models.