DeepInfer

DeepInfer is the large language model inference engine DeepSeek uses to run its API service, released as open source in 2026.

1 articleLast mentioned

DeepInfer is a large language model inference engine developed by the Chinese AI company DeepSeek. It is the engine that powers DeepSeek's own API service, and it was published on GitHub under the Apache 2.0 license in October 2026.

The engine is built on DeepSeek's previously released kernel libraries FlashMLA, DeepGEMM, and DeepEP, and supports expert parallelism and load balancing for MoE models, speculative decoding, and structured output. It targets data-center hardware such as NVIDIA Blackwell-class GPUs and Huawei Ascend NPUs, and the released version supports offline batch inference.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • DeepSeek Opens DeepInfer Serving Engine Alongside MIT-Licensed V4.1-FlashAI News ·

    Three days later, on October 20, it open-sourced DeepInfer under Apache 2.0, the inference engine that actually serves its production API.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.