polysemanticity

The phenomenon in which a single neuron in a neural network responds to several unrelated features.

1 articleLast mentioned

Polysemanticity is the phenomenon in which a single neuron in a neural network responds to several apparently unrelated features or concepts. A neuron that responds to only one feature is called monosemantic.

It has been widely observed in interpretability studies of neurons in convolutional networks and language models, and superposition, in which a model packs more features than it has dimensions, is the main explanation offered for it. Because it makes neuron-level interpretation difficult, methods such as sparse autoencoders, which decompose internal representations into monosemantic features, are being studied.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • Neural Networks Are Already Clean: Study Claims 90% of Neuron Poles Are LegibleAI Security ·

    …a modern language model perform a single, human-readable job, and the messy "polysemantic" neurons that once seemed to prove the opposite were largely misread.

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.