mechanistic interpretability

Mechanistic interpretability is the AI research field that reverse-engineers neural networks into human-understandable features and circuits.

2 articlesLast mentioned

Mechanistic interpretability is an AI research field that reverse-engineers trained neural networks to explain, in human-understandable terms, which features a model represents and how those features combine through circuits to produce its outputs. Unlike approaches that look only at the relationship between inputs and outputs, it analyzes a model's internal activations and weights directly.

It has drawn attention as a way to examine the safety and alignment of large language models. A common experimental method is to locate an internal representation and increase or decrease it to observe changes in behavior.

This entry is based on AIPOST articles and widely known facts. If something is wrong, please send us a correction request.

Articles covering this entry

  • METR Report: AI Progress Doubles in Speed, Raising 2027 Self-Improvement DebateAI News ·

    Existing mechanistic interpretability tools, which attempt to read a model's internal computations directly, proved largely ineffective at detecting true agent…

    Read article
  • Neural Networks Are Already Clean: Study Claims 90% of Neuron Poles Are LegibleAI Security ·

    As an early mechanistic interpretability researcher at OpenAI, he saw a 2023 automated interpretability effort find only about 2 percent of neurons clearly legi…

    Read article

Found a mistake or missing an entry?

Wiki entries are based on AIPOST articles and widely known facts. If an entry is wrong, send us a correction request and we will check and fix it. Names from new articles are added to the wiki after the article is published.