DAPO

An open-source reinforcement learning algorithm for large language models released in 2025 by ByteDance Seed and Tsinghua University, improving on GRPO for reasoning training.

1 articleLast mentioned

DAPO (Decoupled Clip and Dynamic sAmpling Policy Optimization) is a reinforcement learning algorithm for improving the reasoning ability of large language models. It was released in 2025 by the ByteDance Seed team and the Institute for AI Industry Research (AIR) at Tsinghua University, together with its training code and dataset as open source.

DAPO builds on GRPO (Group Relative Policy Optimization), the method used for models such as DeepSeek-R1, and adds four techniques: clip-higher, which decouples the upper clipping bound of the policy update from the lower one to allow more exploration; dynamic sampling, which filters out prompts whose sampled responses are all correct or all wrong and therefore provide no learning signal; a token-level policy gradient loss computed per token rather than per response; and overlong reward shaping, which gradually penalizes excessively long responses. Together these aim to reduce entropy collapse and training instability common in long chain-of-thought reasoning training.

The release reported that a Qwen2.5-32B-based model trained with DAPO reached a higher score on the AIME 2024 math competition benchmark with fewer training steps than previously published methods. It has since become one of the widely used policy optimization methods for reinforcement learning of language models.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Interaction-as-a-Judge: Computer-Use Agents Test AI-Built Games and Web AppsAI Performance ·

    For training-time use, the team ran reinforcement learning on a Qwen 2.5 9B generator using DAPO (Direct Alignment and Preference Optimization) updates.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.