beam search

A decoding algorithm that keeps several candidate token sequences at each step to find an output with high overall probability.

1 articleLast mentioned

Beam search is a decoding algorithm in which a language model keeps a fixed number of high-probability candidate token sequences (the beam width) at each generation step and finally selects the sequence with the highest overall probability. Compared with greedy decoding, which picks only the single most likely token at each step, it can find outputs whose probability as a whole is higher, but tracking multiple candidate states costs more time and memory.

It has long been used in tasks such as machine translation and speech recognition, where there is a relatively well-defined correct output. In conversational LLM services, sampling-based decoding with temperature, top-k, and top-p is more common because of cost and the need for diversity.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Temperature, Attention and Context Rot: Model Mechanics Behind Agent DesignAI in Practice ·

    Candidates in beam search can be ranked by cumulative probability, a task-specific value function or an external verifier.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.