A research essay titled "Neural Networks Are Already Clean," written by interpretability researcher Nick Cammarata and his team, makes a claim that runs against decades of assumptions in AI research: the inside of a neural network is not a hopeless tangle. According to the essay, roughly 90 percent of neuron poles in a modern language model perform a single, human-readable job, and the messy "polysemantic" neurons that once seemed to prove the opposite were largely misread. If the claim survives peer review, it would change the starting point for anyone trying to make AI systems transparent and safe.

The work is explicitly framed as a working hypothesis in an informal essay, not a peer-reviewed result. But the evidence it presents, from re-examined neurons to pruning experiments, is concrete enough to be worth understanding in detail.

Why neural networks became black boxes

Mechanistic interpretability is the field that tries to explain what the internal components of a neural network actually compute, so that humans can trust and steer the model. The difficulty of that task traces back to a design choice made in the 1980s.

At the time, cognitive science and AI debated two ways to store knowledge in a network:

  • Localist representations give each concept its own dedicated unit. One node fires for "cat," another for "car," another for "dog." If the cat node is active, the network is unambiguously thinking about cats.
  • Distributed representations, championed by Geoffrey Hinton, spread each concept across a pattern of activity in many neurons. No single neuron means anything on its own; the concept exists only in the combined pattern, much as a melody exists only in the whole orchestra rather than in any one musician.

Distributed representations won, and for good reasons. They learn features automatically without hand-programming, they let N units encode far more than N concepts through combinations, they degrade gracefully when units are lost, they generalize across shared features, and they support content-addressable memory, where a partial pattern retrieves the whole. The cost was legibility. As networks grew, nobody could say what any individual activation meant, and the "black box" reputation set in. Alan Turing had anticipated this as far back as 1950, writing that we would be largely ignorant of what happens inside intelligent machines.

Superposition appeared to seal the case

Between the 1980s and the 2020s, researchers found something even more confusing when they zoomed in on single neurons: polysemanticity. A single neuron might fire for a fox, a pickup truck, and a high-speed train, items with no obvious relationship.

In 2022, Anthropic published "Toy Models of Superposition," which gave this phenomenon a mathematical explanation. When a layer must represent more concepts than it has dimensions, it cannot keep every concept on its own orthogonal axis. It must either drop rare features or squeeze extra concepts into overlapping, non-orthogonal directions. That overlap makes concepts bleed into one another, producing noise that a small network tolerates in exchange for approximating a much larger one. Because concepts almost always outnumber dimensions, superposition looked practically unavoidable, and much of the safety field came to treat the black box as a structural fact of life.

Cammarata himself shared that pessimism. As an early mechanistic interpretability researcher at OpenAI, he saw a 2023 automated interpretability effort find only about 2 percent of neurons clearly legible.

What changed: agentic audits at scale

The new essay credits a change in tooling. Agentic AI assistants such as Claude and GPT can now run interpretability audits across huge numbers of neurons in a fraction of the time human researchers would need. With that capacity, Cammarata's team went back to the classic examples of polysemanticity and found that the apparent randomness dissolved under closer inspection.

The fox, the truck and the train

The team revisited the well-known neuron that fired for a fox, a pickup truck, and a bullet train. Inspecting where in each image the activation peaked, they found a shared geometric feature: a distinct V-shaped contour. It appeared at the fox's snout and ears, the wedge-shaped hood of the truck, and the aerodynamic nose of the train. The neuron was not juggling three unrelated concepts; it was detecting one shape that happened to show up in three kinds of objects.

Photos of a fox, a vintage pickup truck and a bullet train with red V-shaped lines marking their pointed contours

▲ One neuron detecting a shared V-shaped contour

To test the hypothesis, the researchers examined the neuron's negative pole. Neurons do not just activate; they also push against opposing features. If the positive pole encodes V-shapes, the negative pole should encode their opposite. It did: rounded, O-shaped contours drove the neuron negative. A dog with a pointed muzzle produced a positive activation, while a flat-faced French bulldog produced a negative one.

The essay also reports that even when a network is deliberately compressed to restrict capacity, it does not resort to arbitrary concept mixing. Instead it drops spatial resolution and forms broader categorical splits, for example placing ground vehicles such as fire engines, tow trucks, buses and streetcars on one pole and boats and planes on the other.

Recursive generalization during training

To understand how such structure forms without any upfront plan, the team tracked a single neuron across an entire training run. The progression went in stages:

  1. Early on, the neuron acted like a crude spellchecker, firing only on the letter cluster "iy," as in the name "Miyazaki."
  2. As training continued, it generalized to the acronym "DIY," which still contains "iy."
  3. Later, it fired strongly for the full phrase "Do It Yourself" and for photographs of people assembling furniture with power tools, even with no letters "i" or "y" present.

Cammarata calls this pattern "recursive generalization": a neuron starts with a token fragment, extends to a related symbol, and eventually settles on a broad concept. The essay presents this hierarchical growth, rather than arbitrary packing, as the mechanism behind what earlier tools saw as superposition.

The numbers from the audits

Preliminary audits on Qwen3-32B produced the following breakdown of neuron poles:

Category Share of neuron poles
One primary, identifiable job (often with closely related offshoots) About 90%
Two clearly related jobs About 7%
Uncharacterized or poorly understood 2% to 4%

The essay reports similar legibility when inspecting much larger architectures scaling to trillions of parameters, including Kimi K3.

A second experiment addressed the many weak, noisy activations that make a network look cluttered. The team pruned activations by strength and measured accuracy on ImageNet classes:

Pruning condition Accuracy before Accuracy after
Remove the weakest 75% of activations 72% 71%
Remove the strongest 7% of activations 72% 4%

Grid of mostly dim glowing spheres with a small bright cluster, as tweezers remove dim spheres

▲ Strong activation cluster left after pruning weak ones

The reading offered in the essay is that the bulk of low-magnitude activations are background noise with almost no bearing on the output, while a small set of strong activations carries the real computation. For interpretability, that suggests auditors can filter out weak activations and focus on the strong ones without losing the story of what the model is doing.

Localist and distributed as two views of one system

Rather than declaring a winner in the old debate, the essay argues that localist and distributed representations describe the same architecture at different levels. At the level of individual feature detectors (furry, four legs, barks, wheels, metal), representations are local: each unit has one job. At the level of composite objects (dog, cat, car), representations are distributed: a dog is the pattern furry=1, four legs=1, barks=1, wheels=0, metal=0, while a car is the complementary pattern across the same units.

The resulting picture of a network is an organically grown city: dense, intricate, built without a master plan, yet governed by internal order rather than random entanglement. In hindsight, the complete dismissal of localist ideas in the 1980s may not have been fully justified.

Caveats that still matter

Several limits should temper the enthusiasm:

  • The work is an informal research essay and a working hypothesis. It has not yet gone through rigorous peer review, and it needs to be reconciled with the established mathematics of superposition.
  • Between 2 and 4 percent of neuron poles remain entirely uncharacterized and need deeper study.
  • The percentages depend on categorization thresholds. Compressing a high-dimensional feature space into a few summary statistics means the exact figures can shift depending on how researchers define a distinct feature.
  • The pruning and audit results come from specific models and benchmarks, and whether every finding transfers to every architecture is an open question.

What it could mean for safety, and what to do now

If the AI safety community validates the structure described in the essay, the practical consequences would be significant. Engineers could build precise monitoring tools that trace safety risks through legible circuits, identify and remove computation that wastes energy, and repair broken pathways inside foundation models. Understanding a model well enough to trust it has arguably been the missing half of AI development, and a cleaner interior would make that half far more tractable.

For readers who work on or follow interpretability, the essay suggests several concrete practices:

  • Analyze activations as high-dimensional patterns and inspect the visual or textual contours of trigger inputs rather than relying on high-level labels alone.
  • When a neuron looks polysemantic, examine both its positive and negative poles to find the underlying spectrum it encodes.
  • Filter out low-magnitude activations during audits, since the weakest 75 percent appear to be mostly noise.
  • Trace neurons across training checkpoints to see how abstractions grow from token-level heuristics.

The responsible stance for now is cautious optimism: treat "Neural Networks Are Already Clean" as a strong hypothesis with promising early evidence, watch for independent replication and peer review, and expect the specific percentages to move as definitions are refined.