context bloat

Context bloat describes a large language model's input, or context, filling up with material the current task does not need, such as long conversation histories, many tool definitions or large documents. It increases the number of tokens processed, which raises cost and latency, and it can lower answer quality as the model loses track of what matters.

It is common in AI agents connected to many tools, where tool descriptions can take up a large share of the context window. Typical remedies include loading only the tools and information needed for the current step and summarizing or removing older history. The related decline in model performance as inputs grow longer is called context rot.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.