greedy decoding

The simplest language model decoding method, which picks the single most probable token at every step.

1 articleLast mentioned

Greedy decoding is the simplest decoding method for text generation: at each step the language model selects the single token with the highest probability. It is deterministic, producing the same output for the same input, and is effectively equivalent to setting the temperature to zero.

Because the best choice at each step is not necessarily the best for the sentence as a whole, it tends to produce repetitive or monotonous text. Beam search and sampling techniques such as top-k and top-p are used to compensate.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Temperature, Attention and Context Rot: Model Mechanics Behind Agent DesignAI in Practice ·

    Low values sharpen the peak and push behavior toward greedy decoding; high values flatten the distribution and lift unlikely tokens.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.