scaling law
Scaling laws are empirical relationships showing that AI model performance improves predictably as model size, data and compute increase.
Scaling laws in machine learning are empirical relationships showing that loss or performance improves along regular curves, often power laws, as a model's parameter count, training data and training compute increase.
The concept took shape in research on large language models, notably a 2020 paper by OpenAI researchers and a 2022 paper by DeepMind researchers known as Chinchilla. Scaling laws are used to decide how to split a fixed compute budget between model size and data, and whether the same relationships hold in other domains has to be checked separately.
This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.