stop token

Special token that marks where a language model should stop generating; emitting it ends the response.

1 articleLast mentioned

A stop token is a special token in a language model's vocabulary that marks the end of a text. Also called the end-of-sequence (EOS) token, it causes the inference system to halt generation and return the response as soon as the model emits it. Each model family uses its own notation, such as `<|im_end|>` in ChatML or `<|eot_id|>` in the Llama family.

A base model that has only been pretrained never learns to stop, because its training data is a continuous stream of text, so it keeps writing until the maximum token limit is reached. Appending a stop token to every assistant response during supervised fine-tuning teaches the model where an answer ends and how to respect conversational boundaries. Stop sequences, user-specified strings that halt generation in an API, serve the same purpose.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Pre-Training, Fine-Tuning and RL: Where Frontier LLMs Get Their QualityAI Performance ·

    …ons, tens or hundreds of thousands of them, written by human annotators and synthetic pipelines, in which every assistant turn ends with an explicit stop token.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.