superposition

An interpretability term for the phenomenon in which a neural network packs more features than it has dimensions into overlapping directions.

1 articleLast mentioned

Superposition is an interpretability term for the phenomenon in which a neural network stores more features than a layer has dimensions by packing them into directions that are not fully orthogonal to one another. When the number of features exceeds the number of dimensions, the model compresses rarer features into mutually interfering directions, which is offered as an explanation for polysemanticity, in which a single neuron responds to several features.

The concept became widely known through Toy Models of Superposition, published by Anthropic in 2022, which analyzed the phenomenon mathematically in small models. Methods such as sparse autoencoders, which decompose internal representations to separate the packed features, are being studied. In physics, the term refers to a quantum state in which multiple states coexist.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Neural Networks Are Already Clean: Study Claims 90% of Neuron Poles Are LegibleAI Security ·

    Because concepts almost always outnumber dimensions, superposition looked practically unavoidable, and much of the safety field came to treat the black box as a…

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.