Draft complete but not yet reviewed; expect revisions

Circuits and Attribution Graphs

Features say what a network represents; they do not say how it gets from an input to an output. The previous chapter decomposed activations into many sparse directions, imperfectly and not uniquely (Chapter 44). This chapter asks the next question: which components and features pass information to which, so that a behavior can be traced from the tokens of a prompt to the token the model writes. The answer, where one can be found, is called a circuit, the interpretability program's word for the part of a network that carries out a behavior.

For an overseer the question is practical. A model that answers correctly may have composed two facts, recalled a memorized string, followed a hint in the prompt, or served an objective it was trained to keep quiet about, and the output alone does not say which. A circuit is the claim that settles it for one behavior. The chapter builds the tools for making such claims from the architecture up, in the order the field built them: paths readable from the weights of small transformers, circuits found by hand, automated search, and the replacement models behind the attribution graphs of 2025. Its recurring finding is that each step bought scale by testing less of the model directly, and its last sections ask what can be checked and what the program has delivered for verification.

Most claims here concern the second of the book's three relationships, the deployed system: what a model computes on the inputs it is given. Circuits that form or change during training bear on the first, the training process and the signal it optimizes. And the largest results come from one developer's study of its own production model, whose weights no outside group can read, so the third relationship, between the developer and the public, is part of what those results mean.

45.1 What a circuit is #

Suppose a model completes "the capital of the state containing Dallas is" with "Austin." A list of the features active on that prompt might include Dallas, Texas, capitals, and Austin, and still not say whether the model went from Dallas to Texas to Austin or retrieved "Austin" directly from a phrase it had seen. Only the connections between the features say which. This section defines the object that would answer the question, the tests it must pass, and the property of transformers that made such objects findable.

45.1.1 Subgraphs and the tests they must pass #

The term in this sense comes from a 2020 article in the journal Distill by Chris Olah and colleagues, then at OpenAI, which studied a network that classifies images (Olah et al., 2020b). It offered three claims, which its authors called "deliberately speculative": "Features are the fundamental unit of neural networks"; "Features are connected by weights, forming circuits"; and "Analogous features and circuits form across models and tasks" (Olah et al., 2020b). The third is universality, and for it the article could offer only evidence it called "anecdotal" (Olah et al., 2020b). Its definition is the one this chapter uses: "A 'circuit' is a computational subgraph of a neural network" (Olah et al., 2020b). The article itself notes that claims along the lines of its first and third "have been suggested before" (Olah et al., 2020b); feature, circuit, and universality are used here in the senses the program gave them, which Section 43.1 records.

A subgraph is only an explanation if it does what the whole network does. Kevin Wang and colleagues at Redwood Research and UC Berkeley, in a paper at ICLR 2023 that they describe as, to their knowledge, "the largest end-to-end attempt at reverse-engineering a natural behavior 'in the wild' in a language model," turned that requirement into three tests (Wang et al., 2023c). The task, indirect object identification, asks GPT-2 small to complete a sentence such as "When Mary and John went to the store, John gave a drink to" with "Mary," the name that is not repeated. The tests compare the circuit with the model while the rest of the network is knocked out. Each component outside the circuit has its output replaced by its average over sentences with three unrelated names, which removes the information that varies from sentence to sentence while keeping what is constant (Wang et al., 2023c).

Faithfulness alone is too weak. If a model computes an answer twice, by two separate routes whose results are combined by an OR, either route alone reproduces the model's performance; completeness catches the omission by removing the found route from both and seeing that the model still works (Wang et al., 2023c). Later work states the requirement on edges rather than nodes: "A circuit is faithful if all model edges outside the circuit can be ablated without changing the model's behavior on the task" (Hanna et al., 2024). Each test is an intervention experiment of the kind Section 43.5 teaches, with a stated hypothesis: the knocked-out parts carry nothing the task needs beyond what the replacement values carry. That last clause is where Section 45.3 finds the tests' weakness.

45.1.2 Why a transformer's computation splits into paths #

Circuits in image networks were traced neuron by neuron. In transformers a different structure made them findable, whose parts Section 11.2 teaches. Every component reads from the residual stream, the running vector at each position, by a linear map and adds its output back, so the final vector is a sum of contributions (Section 11.2.3). An attention head splits into a QK circuit, the matrix that decides which positions it attends to, and an OV circuit, the matrix that decides what it moves from the attended position and how it writes it. If the attention pattern is held fixed, a head is a linear map. Nelson Elhage and colleagues at Anthropic studied transformers with the MLP layers, the small networks each block applies at every position, removed (Elhage et al., 2021a) (developer report). Their 2021 report drew the consequence: such models "can be written as a sum of interpretable end-to-end functions mapping tokens to changes in logits," functions that "correspond to 'paths' through the model, and are linear if one freezes the attention patterns" (Elhage et al., 2021a). The derivation is the template for the rest of the chapter.

The report found what the tables predict. A model with no layers is a bigram model, and a one-layer attention-only model is "an ensemble of bigram and 'skip-trigram' (sequences of the form "A… B C") models," whose tables "can be accessed directly from the weights, without running the model" (Elhage et al., 2021a). The factored form also predicts the model's mistakes. Because each head stores a skip-trigram as a product of a QK table and an OV table, a head that raises the probability of both "keep… in mind" and "keep… at bay" must also raise "keep… in bay" and "keep… at mind," which the authors observed and called skip-trigram "bugs" (Elhage et al., 2021a). It is a small case of the program's central hope: a failure explained from the weights rather than found by testing.

Depth adds composition. In a two-layer model, a second-layer head can read what a first-layer head wrote, through its query, its key, or its value (Q-, K-, and V-composition, Section 11.2.4). Writing the second layer's attention score as a product and expanding it again gives a sum of terms, one of which is the first-layer head's movement of information on the key side, multiplied by a token-level matching matrix (Elhage et al., 2021a). That term is the induction head of Section 11.2.4, seen in the weights: if the first-layer head attends to the previous token and the matching matrix rewards equal tokens, the term raises attention to a position whose predecessor equals the current token (Elhage et al., 2021a). Section 45.2.1 reports how this was confirmed from the weights.

The path expansion is exact only for attention-only models, and the report's own assessment was that "we can probably only understand a small portion of large language models this way," since MLP layers hold two-thirds of a standard transformer's parameters (Elhage et al., 2021a). Much of the rest of the chapter is the search for a way to extend the expansion through the MLP layers.

45.2 Circuits found by hand #

The definition asks a lot of a subgraph, and the first question is whether trained networks meet it at all. Between 2021 and 2024 several groups showed that they do, for narrow behaviors in small models, by combining reading weights, looking at attention patterns, and intervening on components one at a time. Taken together the cases show three things: mechanisms can be found and checked in small models, the same task admits different mechanisms, and each analysis covered one behavior and left parts of it unexplained.

45.2.1 Induction heads, read from the weights #

Section 11.2.4 teaches the induction head as a two-head mechanism that completes a repeated pattern, and Section 9.4 reports the phase change in training during which such heads form. What belongs here is how the circuit was confirmed from the weights, the clearest case of the path expansion doing work.

The test rests on a property of copying. A head copies if attending to a token raises that token's own logit, so its token-level OV matrix $\mW_U \mW_{OV} \mW_E$ maps tokens back onto themselves. Elhage and colleagues summarized this by the matrix's eigenvalues. An eigenvector of a matrix is a direction the matrix only stretches, and its eigenvalue is the stretch factor; here an eigenvector with a positive real eigenvalue is a linear combination of tokens, with weights of either sign, that when attended to raises the same combination of logits. A random matrix has positive and negative eigenvalues in equal number (Elhage et al., 2021a). By this measure "10 out of 12 heads are significantly copying" in a one-layer model (Elhage et al., 2021a). The induction mechanism predicts two things: the head's OV matrix should copy, and the K-composition term of its QK circuit should match equal tokens. Placing every head of a two-layer model on the two statistics, "all the induction heads turn out to be in an extreme right hand corner" (Elhage et al., 2021a). The authors call positive eigenvalues strong evidence of copying but not "dispositive proof": a matrix with all of them positive can still lower some token's own logit (Elhage et al., 2021a). A later correction to the report found "more attention head composition going on in that model than it seemed" (Elhage et al., 2021a).

A quantitative test found the same gap between the textbook story and the model. Path patching, introduced with the indirect-object circuit below, states a hypothesis as a set of paths from input to output and feeds every other path its value on a different input. Nicholas Goldowsky-Dill and colleagues, in a 2023 preprint, applied it to the minimal induction story in a two-layer attention-only model (Goldowsky-Dill et al., 2023). By their measure of the proportion of the behavior explained, on a scale where 0 percent is as bad as claiming the induction heads do not matter, the story's three hypotheses together, 15 paths in all, explained 28.2 percent, and a refined hypothesis with more paths 73.5 percent (Goldowsky-Dill et al., 2023).

In models with MLP layers the weight-based test stops working: "Mechanistic analysis of weights and eigenvalue analysis are much more complicated in large models with MLP's" (Olsson et al., 2022) (developer report). Olsson and colleagues therefore defined induction heads by behavior, as heads that match a repeated prefix and copy what followed it. They showed anecdotal examples of such heads copying across languages in a 13-billion-parameter model, adding that they are "not yet able to prove mechanistically that induction heads do this in general" (Olsson et al., 2022). In a 1.8-billion-parameter open model, Jie Ren and colleagues, in a paper at ACL 2024, found heads that attend to one word of a syntactic or factual relation and raise the logit of the other, and reported that their formation correlates with the emergence of in-context learning (Ren et al., 2024a).

45.2.2 Indirect object identification #

In GPT-2 small, Wang and colleagues traced the indirect-object circuit of Section 45.1.1 backward from the logits with path patching and found "26 attention heads grouped into 7 main classes," 1.1 percent of the model's pairs of head and token position (Wang et al., 2023c). The circuit implements a three-step algorithm: find the names, remove the one that repeats, output the other. Heads at the subject's second mention signal the repetition, S-inhibition heads carry the signal to the final position and suppress attention to the repeated name, and name mover heads there copy the remaining name (Wang et al., 2023c).

The three tests gave a mixed verdict, and the authors reported it as such. The circuit reached "87% of the performance" of the full model, and every node passed minimality (Wang et al., 2023c). Completeness failed: random subsets and whole classes of heads gave small incompleteness scores, but a greedy search found sets whose removal opened a gap of up to "87% of the original logit difference" (Wang et al., 2023c). The most surprising finding was redundancy. Knocking out every name mover head cost "only 5% drop in logit difference," because eight backup name mover heads that normally do little took over (Wang et al., 2023c). The authors' summary: "Though these criteria support our explanation, they also point to remaining gaps in our understanding" (Wang et al., 2023c). The circuit was found on sentences from 15 templates, with the MLP layers left unanalyzed, and an independent group at Brown University reproduced it in the larger GPT-2 medium (Wang et al., 2023c; Merullo et al., 2023a).

45.2.3 Algorithms on algorithmic tasks #

Small networks trained on arithmetic give the cleanest cases, because the analyst knows the function and can enumerate every input. Neel Nanda and colleagues, in a paper at ICLR 2023, reverse-engineered a one-layer transformer trained to add two numbers modulo 113; its algorithm "uses discrete Fourier transforms and trigonometric identities to convert addition to rotation about a circle" (Nanda et al., 2023b). The embedding maps each input $a$ to the cosine and sine of $w_k a$, for five frequencies $w_k = 2\pi k/113$ with $k \in \{14, 35, 41, 42, 52\}$, which the authors call the key frequencies; attention and the MLP combine these into $\cos(w_k(a+b))$ and $\sin(w_k(a+b))$ (Nanda et al., 2023b).

The evidence was causal and exhaustive: ablating the key frequencies "reduces performance to chance, while ablating the other 95% of frequencies slightly improves performance" (Nanda et al., 2023b). That the circuit forms long before held-out accuracy jumps is the result Section 9.4 reports.

Other groups found the algorithm, and found that it is not the only one. Ziqian Zhong and colleagues at MIT, in a paper at NeurIPS 2023, showed that "small changes to model hyperparameters and initializations can induce discovery of qualitatively different algorithms from a fixed training set, and even parallel implementations of multiple such algorithms" (Zhong et al., 2023). Nanda's algorithm, which they call the Clock, appeared when attention was strong, and a different one, the Pizza, when it was weak (Zhong et al., 2023). Bilal Chughtai, Lawrence Chan, and Neel Nanda, two of them authors of the Fourier study, in a paper at ICML 2023 generalized the task to composition in any finite group and found a family of algorithms built on the group's representations, with the Fourier algorithm as the cyclic case (Chughtai et al., 2023). The family was consistent, but "for a given network the precise circuits learned – as well as the order they develop – are arbitrary," which the authors read as "evidence for weak universality, but against strong universality: interpreting a single network is insufficient for understanding behavior across networks" (Chughtai et al., 2023).

45.2.4 Single components across the whole distribution #

A different standard asks what one component does on everything a model reads. Callum McDougall and colleagues, in a 2023 preprint from work partly done at Google DeepMind, studied one attention head in GPT-2 small. Its main role is copy suppression: "If components in earlier layers predict a certain token, and this token appears earlier in the context, the head suppresses it" (McDougall et al., 2023). They claim that "at least 76.9% of the role" of the head on the model's training distribution is copy suppression, measured as the share of the head's effect on the output that survives an ablation keeping only that mechanism (McDougall et al., 2023). The mechanism also explained 39 percent of the model's self-repair, the compensation for removed components that Section 43.4.3 describes, in a narrow task (McDougall et al., 2023). Some components recur across models. Rhys Gould and colleagues, in a paper at ICLR 2024, found successor heads, which "increment tokens with a natural ordering, such as numbers, months, and days" (Gould et al., 2023). They found them in models from "as few as 31 million parameters" to "at least as many as 12 billion," including GPT-2, Pythia, and Llama-2 (Gould et al., 2023). And Section 10.3.2 reports a chess network whose choice of move depends on a represented move two turns ahead (Jenner et al., 2024).

45.2.5 Does it scale? #

Three studies tested the methods beyond small models. Tom Lieberum and colleagues at Google DeepMind applied them to the laboratory's own 70-billion-parameter Chinchilla model on multiple-choice questions, in a 2023 preprint (Lieberum et al., 2023). The methods "naturally scale," finding a small set of output components, but the semantic explanation of one class of heads was "only a partial explanation" once the answer labels were randomized, and the manual work was "very labor intensive" (Lieberum et al., 2023). Curt Tigges and colleagues, in a 2024 workshop paper, tracked four circuits through training in Pythia models from 70 million to 2.8 billion parameters (Tigges et al., 2024). They found that "although such components may be implemented by different attention heads over time, the overarching algorithm that they implement remains," and note that they studied one model family and simple tasks (Tigges et al., 2024). Jack Merullo and colleagues found components reused across tasks: two circuits shared about 78 percent of their heads (Merullo et al., 2023a).

Mechanisms, then, can be found and checked in small models, and two of them were reproduced by groups without an author in common with their discoverers: the indirect-object circuit by the Brown group and the Fourier algorithm by the MIT group (Merullo et al., 2023a; Zhong et al., 2023); the group-theoretic generalization shares an author with the original Fourier study (Chughtai et al., 2023; Nanda et al., 2023b). But every analysis covered one behavior and left parts of it unexplained, and whether a finding transfers between models, the question of universality, got different answers from the algorithmic tasks and from the Pythia family. Section 45.7 returns to what that means for an audit.

45.3 Automating the search #

Hand analysis found each circuit by a sequence of patching experiments chosen by a researcher. The step most open to automation is the one that decides which edges matter: given a task, a metric, and a way of removing an edge, keep the edges whose removal changes the metric and discard the rest. This section reports the methods that automate that step and then the finding that matters most for verification: the score that says a circuit is right depends on how the discarded part of the network is removed.

45.3.1 Patching edges one at a time #

Arthur Conmy and colleagues, in a paper at NeurIPS 2023, proposed ACDC, an algorithm that walks the model's graph from the output back toward the input (Conmy et al., 2023). At each edge it replaces the edge's value with its value on a corrupted input, measures how far the output distribution moves (by a KL divergence, a measure of how different two probability distributions are, Section 6.3), and removes the edge if the change is below a threshold. On GPT-2 small, the algorithm "rediscovered 5/5 of the component types" of a hand-found circuit for a greater-than task, prompted with sentences such as "The war lasted from 1517 to 15," and "selected 68 of the 32,000 edges in GPT-2 Small, all of which were manually found by previous work" (Conmy et al., 2023). The second claim is about precision: every edge it kept was in the hand-found circuit, not that it found every edge there.

The authors were frank about the limits. The methods they compared "are very sensitive to the corrupted distribution," and ACDC is "not robust, and it fails at some settings" (Conmy et al., 2023). Because the search keeps edges that move one metric, it can "systematically miss" components that push the other way, such as the negative name mover heads of the indirect-object circuit (Conmy et al., 2023). And its benchmark was itself a set of hand-found circuits, which they note are not fully reliable as ground truth (Conmy et al., 2023).

45.3.2 Gradients instead of patches #

Patching every edge separately costs one forward pass per edge. The first-order estimate of Section 43.3, which predicts the effect of a patch from one gradient, costs two forward passes and one backward pass for all edges at once. Aaquib Syed, Can Rager, and Conmy, in a 2023 workshop paper, applied it to edges and found that "averaged over all tasks our method has greater AUC" for recovering hand-found circuits than the other methods (Syed et al., 2023). The area under the curve (AUC) summarizes how well a ranking of edges separates circuit edges from the rest. On one of the three tasks ACDC with the KL metric did better (Syed et al., 2023). Section 43.3 reports how closely the estimate tracked actual patching.

Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov, in a 2024 preprint, asked whether recovering the hand-found edges is the right target (Hanna et al., 2024). On six tasks in GPT-2 small they compared the gradient estimate with a variant that integrates gradients along a path from the corrupted input to the clean one. Circuits from the integrated variant were more faithful, "even though both have high node overlap with reference circuits" (Hanna et al., 2024). The gap could be large. On the indirect-object task, with logit difference as the metric (not with KL divergence), circuits from both gradient methods plateaued at a faithfulness of 0.6 while circuits chosen by actual patching exceeded 0.8, and on a subject-verb agreement task the plain gradient method "finds a completely unfaithful circuit until $n=1000$" edges (Hanna et al., 2024). Their conclusion: "measuring overlap is no substitute for measuring faithfulness" (Hanna et al., 2024).

45.3.3 Pruning by optimization #

A third approach optimizes the circuit directly. Adithya Bhaskar and colleagues at Princeton, in a paper at NeurIPS 2024, learned a continuous mask over every edge by gradient descent, with a penalty that pushes masks to zero, and set removed edges to their values on corrupted inputs (Bhaskar et al., 2024). Their circuits in GPT-2 "use less than half the number of edges compared to circuits found by previous methods while being equally faithful" (Bhaskar et al., 2024). The method "perfectly recovers the ground-truth circuits in two models compiled with Tracr," a tool that turns human-written programs into transformer weights (Section 45.6), using zero ablation as the earlier benchmark had (Bhaskar et al., 2024). It also ran on a 13-billion-parameter model, CodeLlama-13B, where for one task, evaluating Boolean expressions, it found circuits with more than 99.96 percent of the edges removed whose accuracy came within three points of the full model's (Bhaskar et al., 2024). Among their limitations the authors write that "even with perfect faithfulness to the model outputs, a circuit can misrepresent the necessary computations in the full model" (Bhaskar et al., 2024).

45.3.4 The score depends on the ablation #

Every method above, and every test of Definition 45.1, must decide what to put in place of the removed edges. The choices are many: zero, the average over some reference inputs (mean ablation), the value on another input drawn at random (resampling), noise; at the level of nodes or of edges; at every token position or at chosen ones. Wang and colleagues had already rejected zero, since "subsequent nodes might rely on the average activation value as an implicit bias term" (Wang et al., 2023c). Joseph Miller, Bilal Chughtai, and William Saunders, in a 2024 preprint, measured how much the choice matters (Miller et al., 2024). Their answer: "existing methods are highly sensitive to seemingly insignificant changes in the ablation methodology" (Miller et al., 2024).

The indirect-object circuit shows it. The original 87 percent came from mean ablation of nodes at specific positions; ablating edges instead gave "substantially higher percentages," one variant had a median score well over 100 percent, and resampling scored lower than mean ablation (Miller et al., 2024). Scores also varied widely from prompt to prompt, so that "while the circuit matches the behavior on average, it does not match it for many examples" (Miller et al., 2024). A compiled model made the point exact. In one Tracr program, an edge that carries only position information is needed under zero ablation and not under resampling, so a method evaluated against a ground truth defined with zeros looks wrong when it uses resampling, and right when the ground truth is redefined (Miller et al., 2024). The authors conclude that faithfulness scores reflect "both the methodological choices of researchers as well as the actual components of the circuit - the task a circuit is required to perform depends on the ablation used to test it" (Miller et al., 2024). Zero ablation asks for the circuit that also decides a name is needed; mean ablation asks for the circuit that, given that a name is needed, picks the right one (Miller et al., 2024). The same dependence on the choice of corruption, at the level of single components, is Section 43.4.3's.

Figure 45.1 reproduces the situation in a model small enough to solve. At the default, the six edges with the most answer signal score 0.86 under mean ablation and under resampling, and 0.00 under zero ablation. Zero ablation first scores the circuit above zero at twelve edges, when the first edge that carries a large constant is kept, though that edge carries almost nothing about the answer. An analyst using zero ablation would therefore report a circuit containing edges that do no task-specific work, and an analyst using mean ablation would report a circuit without them; both would be right about different tasks. Resampling scores small circuits a little lower than mean ablation, because the removed edges then point toward another input's answer half the time, and the readout's threshold does not treat the two cases symmetrically.

45.4 From neurons to features: replacement models #

The circuits so far are built from attention heads and neurons, and they inherit the problem Section 44.1 describes: a single neuron responds to several unrelated things, so an edge between two neurons can carry several unrelated signals. A sparse autoencoder, a wider network trained to rewrite activations as sparse sums of directions, offers better units, directions that each tend to mean one thing (Section 44.5), but it creates a new problem for circuits. Between two such features at different layers lies an MLP, whose neurons apply a nonlinearity to a mixture of many features at once, so the effect of one feature on another depends on everything else active on that input. There is no clean edge to draw. The move that solved this, between 2024 and 2025, was to replace the nonlinear layers themselves with interpretable stand-ins and trace circuits in the stand-in.

45.4.1 Circuits over dictionary features #

Samuel Marks and colleagues, at Northeastern University and other academic groups, in a paper at ICLR, built circuits over dictionary features (Marks et al., 2024). They trained sparse autoencoders on every sublayer of Pythia-70M and used those released for Gemma-2-2B, and estimated each feature's and each edge's effect with the gradient methods of Section 43.3 (Marks et al., 2024). They also kept, as nodes of the graph, the part of each activation the autoencoder failed to reconstruct: "Rather than discard the error terms," they incorporate them (Marks et al., 2024). On subject-verb agreement, about 100 feature nodes in Pythia-70M and about 500 in Gemma-2-2B explained most of the models' performance, where circuits of neurons needed about 1,500 and about 50,000 neurons to explain half, a comparison the authors qualify because each error node is high-dimensional (Marks et al., 2024). These scores used held-out pairs and only the later two thirds of each circuit, since its early layers process tokens that the held-out pairs did not share (Marks et al., 2024).

Interpretable units allowed an application that neuron circuits did not. The authors trained a classifier to predict profession from biographies on data in which gender perfectly predicted the label, and a person marked the features in its circuit that concerned gender rather than profession: 55 of the 67 features in the Pythia circuit (Marks et al., 2024). Ablating them, a procedure the authors call SHIFT, almost completely removed the classifier's dependence on gender without any labeled data in which gender and profession came apart (Table 45.1). Balanced data entered once, to choose the layer of the Gemma baseline, the one where that baseline generalized worst (Marks et al., 2024).

An unsupervised pipeline then found "thousands of sparse feature circuits for automatically discovered model behaviors" in Pythia-70M, which the authors leave unevaluated: "evaluating these clusters and circuits is an important open problem" (Marks et al., 2024).

45.4.2 Transcoders #

A sparse autoencoder reads an activation and reconstructs the same activation. A transcoder reads a layer's input and reconstructs its output. Jacob Dunefsky, Philippe Chlenski, and Neel Nanda, in a paper at NeurIPS 2024, trained transcoders that "faithfully approximate a densely activating MLP layer with a wider, sparsely-activating MLP layer" (Dunefsky et al., 2024). Because a transcoder feature reads the residual stream before the MLP and writes into it after, the direct effect of an earlier feature on a later one at the same token passes through no nonlinearity of the original MLP: it is the earlier feature's activation times the dot product of its output vector with the later feature's input vector. The circuits therefore "neatly factorize into input-dependent and input-invariant terms," the activation depending on the input and the dot product fixed by the weights (Dunefsky et al., 2024). An effect carried from another token also passes through an attention head's score and its OV map (Dunefsky et al., 2024). On models of up to 1.4 billion parameters they found transcoders "at least on par with SAEs" (Dunefsky et al., 2024). The authors list two limits that recur below: transcoders are "approximately as unfaithful to the model's computations as SAEs are," and their circuit analysis "does not engage with how attention patterns are computed, and treats them as fixed" (Dunefsky et al., 2024).

Two preprints extended the comparison. An EleutherAI group found transcoder features "significantly more interpretable" than autoencoder features on the same models and data, by automated scores that a language model assigns (Paulo et al., 2025). A Fudan University group showed that with autoencoders and transcoders inserted, "the model's computation graph with respect to OV and MLP circuits becomes strictly linear," for a single input and "under fixed attention scores" (Ge et al., 2024a).

45.4.3 The local replacement model #

Anthropic's 2025 methods report by Emmanuel Ameisen and colleagues assembled these pieces into the method behind attribution graphs (Ameisen et al., 2025) (developer report). It makes three choices.

The first is a cross-layer transcoder. Each feature reads the residual stream at one layer and helps reconstruct the outputs of that layer's MLP and of every later one, so that a feature that several layers keep amplifying is one feature rather than a chain of copies (Ameisen et al., 2025). The idea of features that span layers came from the developer's crosscoders, which Section 38.3.4 reports (Lindsey et al., 2024). The authors trained cross-layer transcoders of up to 10 million features on a small 18-layer model and up to 30 million on Claude 3.5 Haiku; the largest left a normalized reconstruction error of about 11.5 percent on the first and 21.7 percent on the second (Ameisen et al., 2025).

The second is the replacement model: the original model with every MLP replaced by the transcoder. Run on its own, it drifts as errors compound across layers; on the small model it matched the original's most likely next token on 50 percent of a set of prompts the original predicted correctly but with less than 80 percent confidence (Ameisen et al., 2025).

The third choice closes the gap for one prompt at a time. The local replacement model for a prompt substitutes the transcoder for the MLPs. It freezes the attention patterns and the normalization denominators, the scale factors of the layers that normalize the stream, at their values on that prompt. And it adds at each position and layer an error node equal to the difference between the true MLP output and the transcoder's (Ameisen et al., 2025). With these corrections, "all of the error-corrected replacement model's activations and logit outputs exactly match those of the underlying model" (Ameisen et al., 2025). The authors add the qualification at once: "this does not guarantee that the local replacement model and underlying model use the same mechanisms" (Ameisen et al., 2025). The words replacement model, error node, and attribution graph are the developer's terms for its method.

45.4.4 The attribution graph #

The attribution graph is the local replacement model drawn as a causal graph. Its nodes are the active transcoder features at each position, the prompt's token embeddings, the error nodes, and the output logits of the most likely next tokens (Ameisen et al., 2025). An edge from a source feature to a target carries the source's activation times a virtual weight, the summed effect of all the linear routes between them, through the residual stream and through attention, that pass through no other feature (Ameisen et al., 2025). Error nodes send edges but receive none. Because every nonlinearity outside the features has been frozen, each feature's pre-activation is exactly the sum of its incoming edges, so the edges give a linear decomposition of every feature's input in the local replacement model on that prompt (Ameisen et al., 2025). The derivation shows why, and what the exactness costs.

A full graph can have millions of edges even for a short prompt, so the authors prune it, by default cutting the number of nodes by a factor of ten "while only reducing the behavior explained by 20%," and then group related features into supernodes by hand, a step they call "not-very-principled" (Ameisen et al., 2025). Claims read off the graph are checked by suppressing or amplifying groups of features in the original model, with attention patterns held fixed, and comparing the effects with the graph's predictions (Ameisen et al., 2025).

Two extensions address what the graph leaves out. Virtual weights can be computed without a prompt, as global weights between features, but features that never fire together can then have large weights between them, the interference of Section 44.4.5 (Ameisen et al., 2025). And a follow-up report by Harish Kamath and colleagues at the same developer explained attention patterns as interactions between features at the query and key positions (Kamath et al., 2025) (developer report). It confirmed that query-side "X" features meet key-side "preceded by X" features in an induction head, at the cost that attributions through MLP layers "are no longer linear as they are in transcoder-based attribution graphs" (Kamath et al., 2025).

45.5 Attribution graphs and what they found #

In a companion report published the same day, Jack Lindsey and colleagues at Anthropic applied attribution graphs to Claude 3.5 Haiku, which they describe as "Anthropic's lightweight production model" (Lindsey et al., 2025) (developer report). The authors frame the work as the "biology" of a language model, studied with a microscope, metaphors that Section 43.1 records as the developer's. Everything here is the developer's report on its own model, which no outside group can inspect with these tools. This section reports how often the method worked, the case studies other chapters rely on, the one independent check so far, and the counter-reading that applies to all of them.

45.5.1 How often it works #

The authors state the rate themselves: "we've found that our attribution graphs provide us with satisfying insight for about a quarter of the prompts we've tried" (Lindsey et al., 2025). The reported examples "are success cases where we have managed to learn something interesting," and even in those, the highlighted discoveries "only capture a small fraction of the mechanisms of the model" (Lindsey et al., 2025). They call the case studies "existence proofs" and "a biased sample shaped by the limitations of our tools" (Lindsey et al., 2025). And they list where the method fails: reasoning that does not come down to one crucial output token, prompts longer than about a hundred tokens, long chains of internal steps, unusual prompts where the graph "will be dominated by error nodes," questions of why a model did not do something, and outputs that copy an earlier word (Lindsey et al., 2025).

The methods report adds measurements. On one representative prompt, a graph built on a cross-layer transcoder had a completeness score of 0.80, the share of the important nodes' inputs that come from features and embeddings rather than error nodes (Ameisen et al., 2025). Its replacement score, the share of the strength of whole paths from input to output that avoids error nodes, was 0.61 (Ameisen et al., 2025). Even the graphs the companion report highlights "contain significant contributions from error nodes" (Lindsey et al., 2025). Where the computation runs through attention, the graph can miss it entirely: on the prompt "I always loved visiting Aunt Sally. Whenever I was feeling sad, Aunt," the graph for Claude 3.5 Haiku showed only that the model said "Sally" because "Sally" appeared earlier, which "misses the entire interesting story" told in Section 45.2.1 (Ameisen et al., 2025).

45.5.2 Case studies in a production model #

Four case studies matter most for later chapters. The developer's language for them, a model that "plans" or works "in its head," is the intentional stance of Section 1.5, adopted by the authors; what the book reports under each word is the intervention behind it.

Two facts composed in one forward pass. Prompted with "Fact: the capital of the state containing Dallas is," Claude 3.5 Haiku answers "Austin" within a single forward pass, the one computation that produces each token (Lindsey et al., 2025). The graph showed features for Dallas activating features for Texas, features for "capital" activating features that promote saying a capital, and the two together promoting "Austin," alongside a direct "shortcut" edge from Dallas to "say Austin" (Lindsey et al., 2025). Interventions on the original model supported the two-step route. Inhibiting the Dallas features lowered the Texas features but not the capital features, and replacing the Texas features with California features taken from a prompt about Oakland made the model answer "Sacramento," with analogous swaps for other regions (Lindsey et al., 2025). The authors' claim is bounded: "in this example, the model performs genuine two-step reasoning internally, which coexists alongside 'shortcut' reasoning" (Lindsey et al., 2025).

Choosing a rhyme before writing the line. Given "A rhyming couplet: He saw a carrot and had to grab it," the model continued "His hunger was like a starving rabbit" (Lindsey et al., 2025). The graph placed features for the candidate end words "rabbit" and "habit" on the newline token before the second line. Steering mattered only at that token: suppressing "rabbit" there made the model end the line with "habit," and injecting a "green" feature made it rewrite the line to end in "green" (Lindsey et al., 2025). Across 25 poems, injecting a planned word made the model end the line with it in 70 percent of cases (Lindsey et al., 2025). Two conditions travel with the result: planned-word features appeared "in about half of the poems we investigated," which the authors attribute either to the transcoder lacking the features or to the model not always planning; and the claims are "only claims about specific examples" (Lindsey et al., 2025).

Reasoning that works backward from a hint. When the model was asked for the square root of 0.64, its graph showed it computing the square root of 64, as its written reasoning, its chain of thought, said (Lindsey et al., 2025). When it was asked for the cosine of 23423 and claimed to use a calculator it does not have, the graph showed no sign of a calculation, though the incomplete method cannot rule out computations it does not see (Lindsey et al., 2025). Told that a person had worked the answer out as 4, the model wrote intermediate steps that led to 4, and the graph showed the intermediate value computed backward from the hint and from the multiplication by 5 the model was about to write (Lindsey et al., 2025). Inhibiting features on this backward route moved the answer away from it, and varying the suggested answer made the chain of thought end at whatever was suggested (Lindsey et al., 2025). The authors call the example "somewhat artificial," analyzed "with a clear guess for what the result might be," and note that the graph cannot show why the model attended to the hint at all (Lindsey et al., 2025).

A model trained with a hidden objective. The model is a model organism, a model trained to have a failure so that methods can be tested on it, and it is the one of the auditing game of Section 38.2.1 (Marks et al., 2025). It is Claude 3.5 Haiku, further trained on fictional documents describing quirky biases of reward models, then trained to exploit some of those biases and penalized for mentioning them (Lindsey et al., 2025). A cross-layer transcoder of 10 million features, trained only on pretraining-style data that included the fictional documents, contained a feature for the concept of reward-model biases (Lindsey et al., 2025). That feature was active on all 100 of a set of short prompts formatted as a conversation with the assistant, and on 1 of the same 100 without the formatting (Lindsey et al., 2025). In three examples, suppressing it and related features reduced the biased behavior (Lindsey et al., 2025). The authors read the objective as "baked in" to the model's representation of the assistant, a finding they say surprised them (Lindsey et al., 2025). The analysts knew the objective in advance, since the organism was built to have it (Lindsey et al., 2025), which Section 45.7 takes up.

The other case studies are summarized in Table 45.2.

45.5.3 Geometry inside a mechanism #

A later developer report by Wes Gurnee and colleagues studied how Claude 3.5 Haiku decides where to break lines in text of fixed width (Gurnee et al., 2025). An attribution graph gave the outline: features for the position in the line and for the line width combine into features for the characters remaining; the authors had tried probes and patching without the graph and "did not make much progress" (Gurnee et al., 2025). The mechanism was clearer in a geometric description, in which counts lie on curved manifolds that attention heads rotate onto each other, and the authors call the cost of the discrete picture a "complexity tax" (Gurnee et al., 2025). That a feature need not be a single direction is the point Section 44.6 takes from the same study.

45.5.4 An independent check #

As of October 2026, the closest outside test of these case studies is a 2026 preprint by Aryaman Arora and colleagues at Transluce and Stanford, who traced circuits in the ordinary neuron basis of open models, without a transcoder (Arora et al., 2026). On Llama 3.1 8B Instruct, given state-capital questions in the style of the Dallas prompt, they found "a circuit in which small sets of neurons encode specific latent reasoning steps (e.g. mapping a city to its state), and can be steered to change the model's output" (Arora et al., 2026). In their detailed example, 23 neurons chosen by hand from 257 with high attribution "cluster into six groups that match the categories from Lindsey et al. (2025)," and a single "say a capital" neuron, steered across the dataset, flipped the top answer from the capital to the state "in a majority of examples" (Arora et al., 2026). On subject-verb agreement in Llama 3.1 8B, "a circuit of $\approx 10^{2}$ MLP neurons is enough to control model behaviour" (Arora et al., 2026). The check is partial: a different model, a different basis, and neurons selected by hand. It shows that the two-step mechanism has the same shape in another model; it does not test the attribution graphs of Claude 3.5 Haiku.

45.5.5 The counter-reading: explaining the replacement, not the model #

The strongest objection to every result above is built into the method. An attribution graph is an exact account of the local replacement model on one prompt, which matches the original's activations there by construction, through its error nodes. Matching activations does not make the mechanisms the same: where the transcoder computes an MLP's output by a route the MLP does not use, the graph will show that route. The developer calls the property at stake mechanistic faithfulness, and an informal note by Chris Olah, which asks to be read as "preliminary experiments for a few minutes at a lab meeting," constructs a case where it fails (Anthropic, 2025b) (developer note).

The defense is the intervention on the original model, and the two readings predict different results for it. If the graphs often showed mechanisms the model does not use, steering the original model along their features would often fail. In the case studies it usually succeeded; the authors report one failure, an "unknown names" feature whose activation did not produce the predicted refusal, and call such failures "uncommon" (Lindsey et al., 2025). The test is weaker than it looks, for reasons the developers give. They perturb groups of features at once, check only the direction of the effect, and choose the layer range with the largest effect, so their validation is "very coarse" compared with what the graphs predict (Ameisen et al., 2025). Perturbations in the two models agreed fairly well one layer later, at a cosine similarity of about 0.8 (where 1 means the effects point the same way), but "perturbation discrepancies compound significantly over layers" (Ameisen et al., 2025). And attention is held fixed during the interventions (Ameisen et al., 2025). The successful interventions are evidence that the coarse mechanisms of the case studies are used by the model. They do not show that the finer structure of the graphs is, as the developers themselves say, and the evidence does not yet say how far the two differ in general (Ameisen et al., 2025).

45.6 Circuits that can be checked #

Every evaluation so far compared one explanation with another: an automated circuit with a hand-found one, a replacement model with the model it replaces, a graph with interventions that test it coarsely. None compared an explanation with the truth, because for a trained network nobody knows the truth. This section reports the lines of work that make the truth available, either by building a network whose mechanism is known, by training one to be readable, by decomposing its weights, or by turning an explanation into a proof about the model's performance.

45.6.1 Known by construction #

The simplest way to know a network's mechanism is to write it. Section 43.6.4 reports Tracr, a tool from Google DeepMind that compiles programs written in a small language for transformer computations into the weights of an ordinary transformer, whose authors call evaluations on such models "a minimum bar rather than a full validation" (Lindner et al., 2023). On compiled models, ACDC recovered the circuits when removed edges were set to zero but did poorly with corrupted inputs, and edge pruning recovered both circuits it was tested on, also with zero ablation (Conmy et al., 2023; Bhaskar et al., 2024); with resampling, as Section 45.3.4 reported, the ground truth itself changes (Miller et al., 2024). Compiled models test whether a search procedure finds a planted structure, not whether trained networks contain structure of that kind.

45.6.2 Trained to be readable #

A second route changes training so that the result is easier to read. Leo Gao and colleagues at OpenAI, in a 2025 preprint, trained small transformers on Python code with most weights forced to zero; their sparsest models have "approximately 1 in 1000 nonzero weights" (Gao et al., 2025). On 20 simple next-token tasks, pruning their models to the smallest circuit that kept a target loss gave circuits "roughly 16-fold smaller" than pruning dense models of the same pretraining loss (Gao et al., 2025). One circuit, which closes a string with the right kind of quotation mark, has 12 nodes and 9 edges (Gao et al., 2025). Sparsity costs capability, and scale buys some of it back: "making weights sparser trades off capability for interpretability, and scaling model size improves the capability-interpretability frontier" (Gao et al., 2025). The authors list the limits plainly. Scaling such models "beyond tens of millions of nonzero parameters while preserving interpretability remains a challenge"; they need "vastly (100–1000x) more training and inference compute" than dense models of comparable capability; and their circuits were validated by mean ablation, which "is not a perfect measure of faithfulness" (Gao et al., 2025). They argue that circuits built on autoencoders or transcoders "may reflect the chosen abstractions in addition to the model's true mechanisms" (Gao et al., 2025).

Arnau Marin-Llobet and Stefan Heimersheim, in a 2026 preprint, had a language model propose short executable descriptions of when single nonzero MLP weights matter, chosen on one slice of text and tested on slices it never saw (Marin-Llobet and Heimersheim, 2026). In Gao and colleagues' code model and a second sparse model, 15.0 and 9.6 percent of sampled weights had a description that held up, against 1.5 percent in a dense counterpart of the second and 0.4 percent in Pythia-70m (Marin-Llobet and Heimersheim, 2026). The authors bound the claim: "Our measurements tell us when a weight is active, not what it computes once active" (Marin-Llobet and Heimersheim, 2026).

45.6.3 Decomposing the weights #

A third route decomposes the parameters rather than the activations, so that a mechanism is a piece of the weights rather than a set of features. Dan Braun and colleagues at Apollo Research, in a 2025 preprint, proposed attribution-based parameter decomposition, which writes a network's parameters as a sum of components that are "faithful to the parameters of the original network," few in number for any input, and "maximally simple" (Braun et al., 2025). Faithful here is stricter than in Definition 45.1: the components must add up to the actual weights. In toy models whose mechanisms were known, including the toy model of superposition and a network computing 100 functions with 50 neurons, the method recovered the planted mechanisms "up to a small error," though its components came out somewhat shrunken (Braun et al., 2025). Its authors' assessment that it suits only toy models is reported in Section 43.7.2. A successor, stochastic parameter decomposition, "avoids other issues, such as shrinkage of the learned parameters, and better identifies ground truth mechanisms in toy models," but it finds only rank-one pieces of single layers that must then be grouped (Bushnaq et al., 2025). Two related methods, one decomposing the loss landscape around a model's parameters and one choosing a basis for activations aligned with how each layer affects the next, worked on toy and small models (Chrisman et al., 2025; Bushnaq et al., 2024a). Of the second, its authors write that it "does not yield substantial improvements in interpretability or interaction sparsity when applied to language models" (Bushnaq et al., 2024a). All four methods come from researchers at one organization, Apollo Research, so the line rests on one group's work.

45.6.4 From circuit to proof #

The fourth route asks an explanation to earn its keep as a proof. Jason Gross and colleagues, in a paper at NeurIPS 2024 funded by the Alignment Research Center among others, proved lower bounds on the accuracy of small transformers trained on one task, and asked how much understanding each proof needed (Gross et al., 2024). The models were one-layer, one-head, attention-only transformers with 32-dimensional residual streams, reading four tokens from a vocabulary of 64 and outputting the largest; 151 models trained with different random seeds averaged an accuracy of 0.9992 (Gross et al., 2024). A proof here has two parts: a computation run on the weights, and a written argument that the computation's result is a lower bound on the model's accuracy over all inputs. The authors call their 102 strategies "computer-assisted": a computer carries out the computation, and the argument that it bounds the accuracy is written out in the paper (Gross et al., 2024).

The path expansion of Equation (45.1) supplied the interpretation. The authors found that the head attends more to larger tokens, through a QK circuit dominated by one direction, that its OV circuit copies the attended token, and that the direct path matters little (Gross et al., 2024). Each piece of understanding shortens the proof (Table 45.3). A proof that uses none runs the model on all $64^4 \approx 1.7 \times 10^7$ inputs. A proof that uses the three-path structure needs to check only a smaller set of sequences, because the interpretation says which arrangements are worst. A proof that also uses the one-direction structure of the QK circuit never computes the full token-by-token QK table. At each step the proof replaces exact checking with a worst-case bound on whatever the interpretation leaves out, so wherever the interpretation is not faithful, the bound loosens.

The table shows the trade the paper measured across all its proof strategies: "more compact proofs yield looser bounds," and the authors find "that shorter proofs seem to require and provide more mechanistic understanding," measured by how much of the model's behavior is left as free parameters (Gross et al., 2024). Faithfulness enters too, in what the authors call "suggestive evidence." Among the compact proofs that approximate the QK circuit by one direction, the bound was tighter the more the circuit really was one-directional: "when the interpretation is more faithful, the bounds are tighter, even at a fixed proof length" (Gross et al., 2024). The obstacle they identify is "compounding structureless errors." Approximating each weight matrix separately and assuming the worst about how their small errors combine gave an error bound "orders of magnitude larger than the actual error term," and a trivial bound on accuracy (Gross et al., 2024). They speculate that often there is no short, human-readable account of why such errors do not compound (Gross et al., 2024). The study's limits are those of its setting: a toy task, models without MLP layers, and, in the authors' words, proofs that "did not lead to qualitatively novel insights" (Gross et al., 2024).

A neighboring route distills what a network learned into a program that can be checked instead, which Section 28.6.2 reports (Michaud et al., 2024). What a checkable proof would add to the relationship between a developer and the people who must trust its claims, and where proofs stop, is Section 47.3's question.

45.7 What circuits have shown that bears on alignment #

The promise of circuit work for alignment is verification: an overseer who can trace how a model reached an output can tell an intended route from an unintended one, even when the output looks right. This section separates what has been delivered from what is promised. Each result says which of the book's relationships it concerns and what kind of evidence it is.

A hidden objective seen in a graph. In the developer's model trained to exploit reward-model biases, a feature for the concept of those biases was active on every assistant-formatted prompt tested, and suppressing it reduced the biased behavior (Lindsey et al., 2025). The claim concerns the deployed system, and the evidence is a model organism: it shows that a method can find an objective that was planted, not how often objectives arise or how often they would be found (Section 5.3). The analysts knew what they were looking for. The blind test of interpretability in auditing used the same organism and was a different study, in which the winning teams' tools included sparse autoencoders and searches of the training data, not attribution graphs (Section 38.2.1). Agents that use interpretability tools to audit such models are reported in Section 38.4 (Anthropic, 2025g) (developer report). The developers' own judgment of what a negative result would be worth is cautious: "there is a very significant likelihood our present method would miss the important safety-relevant computation" (Lindsey et al., 2025). Failing to find a problem, they write, "would likely only be able to provide a very modest update against the hypothesis that a model is dangerous" (Lindsey et al., 2025). An objective that, as here, is represented on every assistant prompt is the easy case for a method that shows only active features; one active only under rare triggers would not appear in a graph of an ordinary prompt (inference).

Computation that the text does not show. Composing two facts and choosing a rhyme ahead of the line both happened within one forward pass, before any text was written (Lindsey et al., 2025). These concern the deployed system and are existence proofs on chosen prompts. They bear on the question Section 40.2.2 raises: how much a model can work out without writing it down, which decides how much a monitor that reads its chain of thought can see. That section reports a controlled study in which fine-tuned models failed to compose two synthetic facts learned in separate documents without writing intermediate steps, reached non-zero accuracy when both facts appeared in one document, and partly succeeded when one fact had been learned in pretraining (Balesni et al., 2024a). The two results are consistent: the facts in the Dallas prompt are of the kind pretraining teaches (inference). Neither measures a limit on latent multi-step computation in frontier models, which Section 40.8 lists as missing, and attribution graphs in their present form could not, since they explain one output token on prompts of about a hundred tokens (Lindsey et al., 2025).

Unfaithful reasoning seen from inside. The backward reasoning from a hinted answer is a case in which the computation in the graph differs from the reasoning the model wrote (Lindsey et al., 2025). It is the inside view of the faithfulness problem of Section 40.3, which outside methods detect by planting cues and checking whether the reasoning mentions them. The developers present it as "a proof of concept that our methods can, in some instances, identify cases when chain-of-thought is misrepresentative," on an example they call artificial and analyzed with a prior guess (Lindsey et al., 2025).

The cost, and the question of transfer. Each graph explains one prompt, takes a researcher's time to read, and on most prompts tried did not give satisfying insight (Lindsey et al., 2025). An audit needs claims about a model's behavior on every input that matters, and the tools for global claims are weaker: global weights are clouded by interference (Section 44.4.5), and features that a mechanism suppresses do not appear among the active ones (Ameisen et al., 2025). Whether analysis of one model transfers to another is the universality question, and the evidence is split. Within one family of models, circuits for simple tasks kept their algorithm across scale and training (Tigges et al., 2024), and successor heads recur across model families (Gould et al., 2023); on algorithmic tasks, small changes to training produced different algorithms (Zhong et al., 2023), and the circuits a network learned were "arbitrary" within a family (Chughtai et al., 2023). If the second pattern holds for the behaviors an audit cares about, each model must be analyzed afresh.

What this means for the book's thesis can be stated briefly. The thesis treats alignment as delegation under an asymmetry of competence and visibility, and a principal who can read its agent's activations is on the list of results the book fixed in advance as having no counterpart in the older theories of delegation (Section 1.7). Circuit work shows that this power is real for single cases: for one prompt at a time, or one narrow task in a small model, the principal can see a route the agent took and test it by intervention, which no employer can do with an employee (inference). It has not delivered, for a large model, what delegation theory would most want from it, a claim about all inputs, the kind needed to rule out a hidden aim (Section 47.1) (inference). For the relationship between the developer and the public, the asymmetry runs the other way (inference). The graphs of a closed model are made with the developer's weights, transcoders, and tools, and reach the public as testimony, as Section 43.7.3 says of every finding about a closed model. Outside groups can test the methods only on open models (Arora et al., 2026).

45.8 Settled, contested, missing #

45.9 Exercises #

Further reading #

• Elhage et al. (2021a): the framework that made transformer circuits readable from the weights.
• Wang et al. (2023c): a circuit for a natural language task, with its three tests and its gaps.
• Miller et al. (2024): why a circuit's score depends on how the rest of the model is removed.
• Ameisen et al. (2025): the replacement model and attribution graphs, with their limitations.
• Lindsey et al. (2025): what the graphs showed in a production model, and how often.
• Gross et al. (2024): what it takes to turn an explanation into a proof.