DeepInfer is a large language model inference engine developed by the Chinese AI company DeepSeek. It is the engine that powers DeepSeek's own API service, and it was published on GitHub under the Apache 2.0 license in October 2026.
The engine is built on DeepSeek's previously released kernel libraries FlashMLA, DeepGEMM, and DeepEP, and supports expert parallelism and load balancing for MoE models, speculative decoding, and structured output. It targets data-center hardware such as NVIDIA Blackwell-class GPUs and Huawei Ascend NPUs, and the released version supports offline batch inference.