A distributed representation is a way of encoding information in a neural network in which each concept is represented by a pattern of activity across many units, and each unit takes part in representing many concepts. It contrasts with a localist representation, which dedicates a single unit to each concept.
The approach was championed in the 1980s by the parallel distributed processing (PDP) research group, including Geoffrey Hinton. Its advantages include encoding many more concepts than there are units, degrading gracefully when some units are damaged, and supporting generalization because similar concepts share similar patterns. Word embeddings, which represent words as real-valued vectors, are a typical example, and the internal representations of modern deep learning models are mostly distributed. On the other hand, because the meaning of a single unit is hard to read, distributed representations are also cited as a source of difficulty for interpretability.