PPO

PPO is a reinforcement learning algorithm published by OpenAI in 2017 that stabilizes training by limiting how much the policy can change in a single update.

1 articleLast mentioned

Proximal Policy Optimization (PPO) is a reinforcement learning algorithm published in 2017 by John Schulman and colleagues at OpenAI. It belongs to the family of policy gradient methods, which adjust a policy directly in the direction that increases reward, and it stabilizes training by clipping the objective so that the new policy does not move too far from the previous one. It is simpler to implement than the earlier TRPO method while offering comparable stability.

PPO is typically trained together with a value model that predicts the value of a state, which is used to estimate how good an action was. It has been widely used in games and robot control and is also used in reinforcement learning from human feedback (RLHF) to adjust language models.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • AI Agent Tuning: When to Rewrite Prompts and When to Retrain WeightsAI Performance ·

    The older standard, Proximal Policy Optimization (PPO), keeps four full models in memory and trains two of them, including a value model that estimates each tok…

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.