Interpretability is the degree to which humans can understand how and why an AI model produces its outputs; the term also refers to the field that develops methods for improving it. It overlaps with explainable AI (XAI), which focuses on explaining a model's decisions after the fact.
Because neural networks consist of vast numbers of parameters and distributed representations, their internal operation cannot be read directly, so a range of methods are studied: feature attribution, which identifies which parts of an input influenced the output; probing, which examines what information internal representations contain; and mechanistic interpretability, which analyzes internal circuits at the level of individual components. In AI safety, interpretability is regarded as a foundation for detecting and correcting unintended model behavior from the inside.