nanoGPT

nanoGPT is an open-source project released on GitHub by Andrej Karpathy. It implements training and fine-tuning of medium-sized GPT models in a short, readable Python and PyTorch codebase, and includes an example that reproduces a GPT-2 model with 124 million parameters from scratch.

Because the code is compact, it is widely used for learning how language model training works and as a starting point for experiments.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.