top-k

A decoding technique that limits next-token candidates to the k most probable tokens and samples among them.

1 articleLast mentioned

Top-k sampling is a decoding technique in which a language model keeps only the k most probable candidates for the next token and samples among them according to their probabilities. Because the number of candidates is fixed at k, it prevents very unlikely tokens from being chosen while producing more varied output than greedy decoding.

Unlike top-p, which selects candidates by cumulative probability, the number of candidates stays the same regardless of the shape of the distribution. It is usually combined with temperature and top-p.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Temperature, Attention and Context Rot: Model Mechanics Behind Agent DesignAI in Practice ·

    The rule of thumb is task-driven: math and code want low temperature; creative writing wants higher entropy, usually with top-p or top-k.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.