A positional embedding is the component that converts each token's position into a vector and adds it to the input embedding so that a transformer can tell the order of tokens. The attention operation itself is indifferent to input order, so positional information must be supplied separately. It is also called positional encoding.
Approaches include fixed sinusoidal functions, learned vectors for each position, and RoPE, which rotates queries and keys to reflect relative position. Because a model interprets positions correctly only within the range seen during training, the positional embedding scheme determines the practical limit of the context window and how a model can be extended to longer contexts.