InstructGPT is a language model released by OpenAI in January 2022. Built on GPT-3, it was trained with supervised fine-tuning on human-written demonstrations, followed by reinforcement learning from human feedback (RLHF), in which a reward model learned from human rankings of multiple outputs is used to optimize the model. The method is described in the paper "Training language models to follow instructions with human feedback" published the same year.
InstructGPT was found to follow user instructions better than the original GPT-3 while producing fewer untruthful or harmful outputs, and it established the RLHF training approach later used widely in conversational language models including ChatGPT.