Top-k sampling is a decoding technique in which a language model keeps only the k most probable candidates for the next token and samples among them according to their probabilities. Because the number of candidates is fixed at k, it prevents very unlikely tokens from being chosen while producing more varied output than greedy decoding.
Unlike top-p, which selects candidates by cumulative probability, the number of candidates stays the same regardless of the shape of the distribution. It is usually combined with temperature and top-p.