mid-training
Mid-training is a stage of large language model development, between pre-training and post-training, in which the model is further trained on curated data.
Mid-training is a training stage in large language model development that sits between pre-training, which builds general capabilities from vast amounts of data, and post-training, which refines a model through instruction tuning or reinforcement learning. The model continues training on high-quality data selected to strengthen specific abilities such as mathematics, coding, reasoning or long-context processing.
It prepares the base model before post-training, and the data and methods used vary from developer to developer.
This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.