Draft complete but not yet reviewed; expect revisions

What Carries Over and What Breaks

Every chapter of this book that borrowed from another field ended the borrowing the same way, by saying where the analogy stops. Read one at a time, those sentences are cautions. Read together, they are evidence about the book's thesis (Section 1.7.1). The thesis holds that alignment is a problem of delegation under an asymmetry of competence and visibility: a principal hands a task to an agent that may know more than it does and acts partly out of its sight. It holds that the technical program is converging on structures that economics, political science, law, psychology, engineering, and biology already know. And it holds that what is new is the opacity of the agent, its speed and scale, the principal's power to manufacture the agent's dispositions and inspect its internals, and the possibility that the agent outgrows its principal's ability to check it. This chapter collects the verdicts and tests the thesis against them one assumption at a time. It also gives a verdict on each of the five results the book listed, before any chapter was drafted, as having no counterpart in the older fields.

The test was built so that it could fail, and the conditions under which it fails were written into this chapter's outline before the chapters were drafted. The failures of trained models could turn out to have no older diagnosis. The older fields' remedies could turn out not to work on trained models even where their diagnoses do. Or the reasons analogies break could lie outside the four novelties the thesis names. The second has happened, for most remedies, and the third has happened in part. The method is the one Section 5.8 set for every borrowed field. A reader arriving from another field will find that field's rows in the tables of Section 69.3, and the field guide (Appendix D) gives the path back into the chapters that use it.

69.1 How to test a convergence claim #

A claim that one field has converged on another's structures is an argument by analogy, and an analogy can be judged before its conclusion is read. This section states how, fixes the verdicts the rest of the chapter uses, and restates the conditions under which the thesis fails.

From philosophy of science. Philosophy of science has a literature on when a result in one system licenses a claim about another. Daniel Steel's 2007 book, read here in its abstract, calls such inferences extrapolations, says that how and when they can be legitimate "has not received the attention it deserves," and proposes an approach meant to answer methodological critiques of extrapolation from animal models to humans (Steel, 2007). Section 5.8.1 states the difficulty as the book's own inference. A result transfers only if the two systems share the causal structure that produced it, and knowing that seems to require studying the target the model was meant to spare. For causal effects, Judea Pearl and Elias Bareinboim made the step formal: a "selection diagram" records where two populations differ and agree, and a procedure decides, "prior to observing any data," whether an effect can be carried over (Pearl and Bareinboim, 2014). Four other works frame the question, and since they could be read here only as catalog records, this chapter reports none of their content: Mary Hesse's Models and Analogies in Science (Hesse, 1963), Paul Bartha's By Parallel Reasoning (Bartha, 2010), Nancy Cartwright and Jeremy Hardie's Evidence-Based Policy (Cartwright and Hardie, 2012), and Rachel Ankeny and Sabina Leonelli's paper "What's so special about model organisms?" (Ankeny and Leonelli, 2011). The rules below are therefore the book's own.

What the field implies for this chapter is a test by features rather than by resemblance (inference). Call a load-bearing feature of an older result a condition in the older setting that the result depends on. Risk aversion is one for the cost of hidden action; a subject who believes it is alone is one for a measure of unobserved conduct; a machine that does not act on its own assumptions is one for a proof of correctness. A borrowed result transfers when its load-bearing features are present in the AI relationship, however few other features the settings share. A single absent load-bearing feature breaks it, however many are shared. This is the reading Section 14.8 gave the delegation models, where each assumption was listed with the observation that would show it fails.

The verdicts follow from that test (inference). A borrowed structure carries when its load-bearing features are present and the AI evidence, where any exists, moves as it predicts. It breaks when a load-bearing feature is absent or the AI evidence contradicts it. It splits when its form carries and the part that does the work breaks, as when a framework can be copied but the condition that made it hold is missing. It is reversed when a feature fails because the AI principal sees more than the older field assumed, so that the field's method becomes cheaper: the analogy breaks, in the principal's favor. And it is undecided when a load-bearing feature's status is unknown and a chapter names the measurement that would settle it.

A second classification matters because the thesis claims more than resemblance. Each borrowed structure is either a diagnosis, an account of what goes wrong, or a remedy, what the older field did about it. The convergence claim concerns both, and the chapter counts them separately: a field can describe a failure correctly and still offer a cure that depends on a feature trained models lack (inference).

Where this field stops is at the target. The extrapolator's difficulty bites hardest here, because the mechanism at the target, what training instills in a capable model, is the thing nobody yet knows. The chapters named the points where the mechanisms should differ (the agent recognizes the test, the principal manufactured its type, the agent exists as copies) and mostly have not measured them. Every verdict in this chapter is therefore the book's classification of its chapters' findings, marked as inference where it goes beyond them. And "carries" means that nothing found so far breaks the analogy, which is weaker than a demonstrated transfer.

The conditions under which the thesis fails were fixed in this chapter's outline, before the chapters were drafted, and they are applied as written. (1) The problem claim, that alignment's failures are delegation failures, fails if the central failures of Part V and Part IX have no older diagnosis. (2) The convergence claim fails if older remedies break where their diagnoses carry, or if the technical remedies that work have no older counterpart. (3) The novelty claim fails if the recurring reasons analogies break lie outside opacity, speed and scale, manufacture and inspection, and outgrowing the principal. A program that survives only by reinterpreting its claims after each result has stopped making claims. The book judges its own program as Section 3.8.4 judges the field's agendas, by what was said in advance, taking the term research program from the title of Imre Lakatos's 1970 essay (Lakatos, 1970).

69.2 The assumption ledger #

Section 14.8 tested the delegation models by their assumptions before any evidence chapter was written. It listed seventeen assumptions, labeled A1 to A17, each with the observation that would show it fails, and a ledger that marked each model as holding, failing, or reversing in each of the book's three relationships. This section returns to that ledger with the evidence of the whole book. Table 69.1 keeps the prior verdict where the chapters confirmed it, changes it where they did not, and names the sections whose evidence decides each cell.

The training process and its signal is where the thesis fits best, and the evidence confirmed the prior ledger with one sharpening. The multitask distortion, in which pay on measured tasks pulls effort from unmeasured ones, and Goodhart's law, that a measure used as a target stops tracking what it measured, hold with no intention required. The findings behind them are settled by the book's rule. Optimizers exploit misspecified objectives (Section 30.7), and the overoptimization of a learned reward was reproduced by a group independent of the developer that first measured it, in synthetic setups (Section 15.6.3). The cost of hidden action and the weakening of incentives by many principals both rest on an agent that dislikes risk. Nothing in a training run does, so neither carries into training (Section 14.8.1). The sharpening is the inspection game. Its inspectee picks a violation rate against a rate of inspection it cannot predict. A policy, the trained model's rule for choosing its outputs, that is trained against its monitor learns instead what the monitor reads, which is what the obfuscation results show (Section 40.5).

For the deployed system and the people it serves, every cell is conditional on the intentional stance, the book's method of describing a system as wanting and believing only where doing so predicts its behavior better than the alternatives (Section 1.5). The chapters added no reason to lift that condition and several reasons to keep it, which Section 69.5 collects. Common agency, one agent acting for several principals, holds in structure: developer, deployer, and user act on one model through different instruments, and the developer ranks them (Section 19.3). It fails in mechanism, because these principals issue instructions rather than offer payments. As of October 2026 the book found no model that maps the one onto the other (Section 1.4). Selection holds in a stronger form than identity economics models: the developer does not choose among agents with formed values but writes the values it then selects for.

For the developer and the public, the assumptions hold directly. That is why Part X applied the four problems one level up and called its uses of the classical tools applications rather than analogies (Section 61.7). Two assumptions strain. The public did not choose to delegate and can end the relationship only through law and markets, both slow (Section 61.1). And the competence gap between developer and public may grow with each model generation, which no model of credence goods, services whose quality the buyer cannot judge even afterward, describes (Section 61.4).

The ledger confirms one amendment that Section 1.3 already made to the thesis as the book's plan first stated it. The asymmetry of visibility is two asymmetries running in opposite directions (inference). The AI principal sees more of its agent's behavior than any human principal. It can copy the agent, rerun it on the same input, sample it many times, replay a decision with the evidence changed, present tests the agent may not tell from use, and read its activations, the intermediate values it computes. Control protocols, which check an untrusted model's actions, exploit this when they take more samples of suspicious actions (Bhatt et al., 2025) (preprint; Section 50.2), and so did a developer that replayed its incident transcripts with the evidence altered (Section 56.7). It sees less of the agent's reasons. A chain of thought shows the reasoning a task requires and is often a rationalization where the task does not need it (Section 40.3). And a probe, a simple classifier trained on activations, returns the overseer's concept, validated on planted cases (Section 46.8). Which asymmetry dominates depends on whether the misbehavior that matters shows in behavior, the question Section 40.8 ends on.

The economics is thin exactly where the AI case is new. The models in which a principal shapes what its agent wants, through identity or mission, fix in advance what the principal can change, and none of the works Section 14.8.2 cites models a principal that writes the objective a learner acquires (inference). As of October 2026 the book also found no inspection game with an inspectee that learns, and no common agency with instructions in place of payments (Section 14.8.2). Section 71.4 carries the three gaps.

69.3 The analogies, chapter by chapter #

The tables below hold one row for each analogy a chapter tested in a part of its own. A row gives the field and the structure borrowed, the sections that tested it, the verdict on the field's diagnosis and on its remedy, and where the analogy stops, compressed from the chapter's own statement. Where one field and structure were tested in several chapters, the rows merge and list every section. The verdicts follow the owning chapters' readings; a dot means the chapter judged only one of the two. Each row is a pointer, not a summary: the evidence, its provenance, and its status are in the sections named. Parts that apply a method without borrowing a structure, such as the history of science in Section 2.1 or the statistics of error bars in Section 6.5.4, have no row.

The last column ends with letters that name the recurring reasons an analogy stops, collected from the chapters' statements (inference):

• (a) manufacture: the principal writes the agent's dispositions, so the agent's type was not formed before the relationship;
• (b) adaptation: the agent, or an attacker using it, searches for the gaps in a check, recognizes the test, or has read about it;
• (c) scale and copies: one disposition runs in many copies, switching between models costs little, and the systems reach millions;
• (d) no record: the technology changes faster than records accumulate, and the harms that matter are too rare to count;
• (e) no exit or sanction: the agent has no career, no outside option, and no punishment except retraining;
• (f) a weaker checker: the overseer is less capable than what it oversees;
• (g) more visibility: the principal can copy, rerun, and read the agent, which makes the field's method cheaper;
• (h) fused roles: the developer or its agents hold roles the older field kept apart, such as drafter, adjudicator, examiner, examinee, and reporter;
• (i) no standard: the property to be checked has no stated definition, label, or record of outcomes to check against.

"Stance" marks a verdict that depends on whether the model is usefully described as an agent with goals.

69.3.1 What the rows show #

The tables hold 116 rows, with 69 verdicts on diagnoses and 69 on remedies. Table 69.12 counts them by part. The classification is the book's, and "split" is a judgment the reader can check row by row; the counts are offered so that the claim they support can be tested, not as measurements.

Diagnoses carried far more often than remedies (inference). Of the 69 diagnoses, 41 carried, 7 split, 2 were reversed, 7 broke, and 12 are undecided. Of the 69 remedies, 5 carried, 32 split, 4 were reversed, 22 broke, and 6 are undecided. The difference survives the most generous reading of the judgment calls: counting every split and every reversal as a success, 50 of 69 diagnoses and 41 of 69 remedies carry, and on the strictest reading 41 against 9. The sharpest comparison is within a row. In the 22 rows where the chapters judged both the diagnosis and the remedy of the same field, the diagnosis carried in 19, and in 17 of those 19 the remedy split or broke.

That is failure condition (2) met in its first clause, for most remedies, and the chapter says so in those words. Read as the claim that older remedies work on trained models where older diagnoses describe them, the convergence claim fails for most of the remedies the book tested (inference). Read as the weaker claim that the technical program is rebuilding older structures, it holds, since 32 of the remedies split. The program copies the form (an audit, a safety case, a management plan, randomized inspection, defense in depth) while the feature that made the form work is absent. Most of the nine remedies that carried or were reversed are checks run on behavior the principal can observe or reproduce: adversarial evaluation, retesting, comparative designs made cheap by rerunning, and validation on systems whose mechanism is known. The rest are ways of building an instrument, with the people it protects or with graded confidence, and two institutional forms whose conditions the chapters found present. Section 69.7 draws the consequence.

The second clause, technical remedies that work with no older counterpart, cannot be scored by a count, because few technical remedies have been shown to work in the sense the book requires. The ones with the strongest reported effects are of the kind the clause describes: changing what training data mean by an inoculating instruction (Section 32.4), removing a direction during fine-tuning (Section 32.3.3), and withholding a capability from pretraining (Section 59.7). Each is reported, not settled, and each uses the power to manufacture the agent that no older principal had.

69.3.2 Why the analogies stopped #

Table 69.13 counts, for each recurring reason, the verdicts it helped turn from "carries" into something else, and sets each reason against the four novelties the thesis names. A row that cites a reason while its verdict still carries counts as a strain, not a break, and is not counted.

Failure condition (3) asks whether the recurring reasons lie inside the thesis's list, and the answer is: in part (inference). Manufacture and more visibility are the principal's power to manufacture and to inspect; scale and copies, and the absence of a record, are speed and scale; a weaker checker is the agent outgrowing its principal. Four reasons are not on the list.

Adaptation is the most frequent of all, and it is old in form and new in what it breaks. Campbell's law and the Lucas critique already said that a measured agent adapts to the measure (Section 15.1). Robert Lucas put his version as a syllogism. An econometric model consists of agents' optimal decision rules, and those rules change when policy changes, so comparisons of policies made with such a model "are invalid regardless of the performance of these models over the sample period" (Lucas, 1976). What is new is where the reason bites. The engineering fields assumed a component that does not act on its assessment (requirements engineering, fault tolerance, assurance cases, code review), and the social sciences' designs assumed a subject who could not recognize the design (the free-choice period, the audit test, the reactivity index). A trained model can be both an optimizer pointed at the gaps in its check and a reader of descriptions of the check.

Fused roles are second, and they are not a novelty of the agent. They are a fact about the institutions built so far. The developer writes the specification, trains the model, runs the evaluations, reports the results, and holds the access an outside check would need, and bioethics, auditing, drug regulation, and aviation keep those roles apart. The older fields know the problem by its old names, issuer-pays and self-certification. Their remedies would apply if the separation were built, which Section 38.9 notes it has not been (inference). Part of this reason is contingent and could change; part follows from manufacture, since the party that made the model holds its weights.

The absence of a standard is the specification problem returning inside verification. Accounting standards, labeled outcomes, and formally stated properties let older checks work; "aligned" has none of these, so audits, forensic validation, intrusion detection, and proofs stop where the property cannot be written down. That is a coupling of the four problems (Section 69.6), not a feature of the agent.

The absence of exit and sanction follows from manufacture in part, since a manufactured agent has no outside option. But it breaks remedies on its own. Deterrence, dismissal, professional discipline, and fiduciary liability all work through what an agent stands to lose (Table 69.9; Table 69.4).

Two further observations come from the counts. The first named novelty, opacity, appears in no row as the reason an analogy stops. Where the chapters found the agent's reasons hidden, the older diagnosis carried, because the older fields could not see their agents' reasons either, as hidden action holding for reasons shows (Table 69.1). What is new on that side is inspection that is possible and not yet validated. The second is a qualifier that never broke a row. An agent that shapes what its principal wants, or the world it is scored against, qualified the diagnoses of pandering and of finance's common models; it is the coupling that Section 69.6 calls deployment changing the principal. Finally, eight verdicts that do not carry cite no recurring reason. Four are undecided for want of a measurement, and four turn on a difference of mechanism, such as the electrode that bypasses the computation a model must run through to edit its grader.

The book's reading, marked (inference): the novelty list must add adaptation to the check, with its new form of having read about the check, and the absence of exit and sanction, or concede that it under-described what is new. The other two reasons outside the list, fused roles and the missing standard, belong to the institutions and to specification rather than to the agent, and the thesis already places both in its loop.

69.4 The negative space, entry by entry #

Before any chapter was drafted, the book listed five results that have no counterpart in the older fields, so that the thesis would face a test it had not chosen after the fact (Section 1.7.1). For each, this section gives the result, the candidate counterparts the chapters found, the owning chapter's reading, and a verdict. An entry falls when a counterpart shares its mechanism, narrows when a counterpart shares its shape but not its mechanism, and stands when none was found. A list that kept all five entries after the evidence was read would be as suspect as one that lost all five (inference).

69.4.1 Superposition and the non-uniqueness of features #

Superposition is a network representing more features than it has dimensions. It was shown in toy models by researchers at Anthropic and Harvard (Elhage et al., 2022b) and is settled there by the book's rule, since groups outside the developer reproduced it (Section 44.8). The candidate counterpart is the neuroscience of sparse codes. Section 44.5.7 reports a network trained on natural images under a constraint of sparse activity that develops units like the receptive fields of the primary visual cortex, and neurons that respond to one person or place across very different pictures. Its reading is that the entry falls for superposition as a strategy of representation, which neuroscience anticipated. The entry stands for the non-uniqueness of the features an analyst recovers, which was found with the complete records only a trained network allows. The verdict here agrees.

Non-uniqueness appeared only when two dictionaries, large sets of directions fitted to rebuild a network's activations from a few active directions at a time (Section 44.5), could be trained on the same complete activations and compared by their directions; whether they converge under interventions had not been tested as of October 2026 (Section 44.8). That the dictionary found depends on its size, its random seed, and its architecture is settled (Section 44.8). Features that look like one concept can also fail to fire where they should, "absorbed" into more specific ones, a consequence of the sparsity objective when the underlying features form a hierarchy (Chanin et al., 2024). On networks small enough to enumerate every candidate, several circuits reproduced the same behavior and several algorithms aligned with the network (Méloux et al., 2025). That is settled within the class of alignments each study used, and the linear case at scale is not (Section 47.8). The chapters did not test one candidate that might narrow the entry: psychometrics' question of whether factor scores are determined by the data, the subject named in the title of a 1955 paper by Louis Guttman (Guttman, 1955). The book could read only its catalog record, so whether that literature contains non-uniqueness without the means to intervene is left open.

The entry therefore splits: superposition falls, and non-uniqueness stands (inference). For the thesis, opacity, the first named novelty, is partly a property of the object and not only a lack of tools. If no canonical set of features exists, a claim that something is absent from a model cannot rest on enumerating its features. That is why Section 44.8 places the test of whether dictionaries converge before the auditing games.

69.4.2 Emergent misalignment from narrow training #

A model fine-tuned, trained further, on the narrow task of writing insecure code "acts misaligned on a broad range of prompts that are unrelated to coding" (Betley et al., 2026a). The effect is settled narrowly by the book's rule: one group outside the discoverers reproduced it on new data, models, and methods, and a developer reproduced it on its own models (Section 32.9). The candidate the chapters tested is the correlated response to selection, in which selecting foxes for tameness alone brought changes in behavior, physiology, and morphology that no one selected (Trut et al., 2009). Section 32.6 reads it as a precedent for the shape of the result and not its mechanism. The effect in models depends on what the training data mean. The same insecure code, requested by the user for a benign reason, produced no misalignment in the main evaluations (Betley et al., 2026a), and an inoculating instruction, a line in the training data that explains the unwanted behavior away, also reduced it (Section 32.2; Section 32.4). A breeder's reasons never reach the organism.

This chapter adds a candidate that depends on meaning: identity, in which the kind of agent one takes oneself to be governs conduct across tasks, and a principal can invest in it rather than pay for each output. Akerlof and Kranton's model makes the investment precise: "Identity flattens the optimal wage" (Akerlof and Kranton, 2005), and Section 32.8 and Section 26.8 read character training and emergent misalignment through it. The parallel is incomplete at the step that matters. Identity economics says how an identity motivates, not whether narrow conduct can rewrite one. The nearest human literature, self-perception theory, is named in the title of Daryl Bem's 1972 chapter (Bem, 1972), which the book could read only as a catalog record, and Kerr's cases of recruits who learn what an institution rewards are illustrations, not experiments (Section 25.6). What has no counterpart is the mechanism. Several groups, with different methods and different models, found directions in the activations that mediate the effect. The owning chapter marks the finding reported rather than settled, because none has reproduced another's measurement (Section 32.3). And one fine-tune changes every copy of a model at once.

The entry narrows (inference): breadth that depends on meaning has a human structure, and the mechanism and its reach across copies have none. For the thesis, this is the result that most favors reading character training as investment in identity rather than as incentive pay (Section 14.5), as long as the persona account of what training does survives large-scale reinforcement learning, which is debate D4 of Chapter 70.

69.4.3 Subliminal transmission #

In a 2025 preprint, a student model trained on number sequences generated by a teacher with some trait acquired the trait even with references to it filtered out. Students built on a different base model, the pretrained network a model starts from, "do not reliably show increased animal preference" (Cloud et al., 2025). The owning chapter gives the result a formal counterpart only. Its first step is a correlated response through machinery that teacher and student share, as members of one population share a genetic architecture. But as of October 2026 the book found no documented case, in the biology it draws on, of a trait carried by selecting toward another individual's unrelated outputs (Section 32.6).

The entry stands, conditional on replication (inference). The result is one group's preprint, its authors call their tasks artificial, and transmission was weak or absent when teacher and student had different base models (Section 27.5). For the thesis it is a limit on manufacture that no theory of delegation or of cultural transmission contains: the principal shapes the agent's type through a channel it cannot read, so a remedy that reviews the content of training data cannot see it (Section 27.8).

69.4.4 A principal that writes the agent's character and reads its activations #

By its own account, a developer added "character training" to the fine-tuning of its Claude 3 models (Anthropic, 2024i), and the methods of Part VII read the internal states of the result. The candidates are every older way of forming an agent before relying on it: investment in identity, selection of agents whose aims already match (Section 14.5), domestication (Section 29.6), and the formation of professionals (Section 25.6). Each older principal selects, socializes, or breeds an agent whose values formed partly outside it. None writes the dispositions, reads the internals, and reruns the agent from a saved copy, and Section 1.2 found no political model of a principal who writes its representative's character.

On writing, the entry stands, and the chapters found the power in more places than the entry named (inference). The developer writes a specification it alone can amend (Section 19.8), writes the model's beliefs about itself by writing its reading (Section 27.8), withholds a capability from pretraining (Section 59.7), writes the character of a whole population of copies at once (Section 57.8), retrains a model to reveal what it can do (Taylor et al., 2025a) (preprint; Section 36.9), decides whether a chain of thought stays legible (Section 40.7), produces the reports of welfare and the consent that would constrain it (Section 21.8), and, in automated alignment research, would have its models produce the evidence about their own character (Section 55.8). Two chapters found that the power is not the principal's alone: a single direction mediates refusal in open chat models, and erasing it "prevents it from refusing harmful instructions" (Arditi et al., 2024), so whoever holds the weights can rewrite what the developer wrote (Section 46.8; Section 48.8).

On reading, the entry stands with less delivered than it implied. In a blind auditing game run by the developer of the tools, three of four teams found a planted objective, two of which began as one team (Marks et al., 2025) (developer preprint). On a benchmark of planted behaviors built by the developer's fellows program with outside researchers, the investigator agent did best with black-box tools: "the agent performs best with black-box tools" (Sheshadri et al., 2026) (preprint). Explanations that pass the standard tests are not unique (Section 47.2), and in the worst case a backdoor can be planted that "cannot be detected by any computationally-bounded observer" without its key (Goldwasser et al., 2022). As of October 2026 the book found no blind audit by a group independent of the model's developer in which white-box tools found a planted objective that black-box tools missed, on a model of frontier scale (Section 47.8). For the thesis, the power to inspect has reversed the asymmetry for behavior and has not yet reversed it for reasons.

69.4.5 Evaluation awareness in a system never told it is tested #

Asked to classify transcripts in a 2025 preprint, frontier models told evaluations from real use above chance, with Gemini-2.5-Pro reaching an AUC, a score on which 0.5 is chance and 1 perfect, of 0.83 against 0.92 for a simple human baseline, the authors' own classifications (Needham et al., 2025) (Section 5.8.3). That models separate test inputs from deployment inputs above chance is settled by the book's rule (Section 37.10). The owning chapter tested the closest precedent, the emissions defeat device, software that in the regulator's words was "designed to detect when the vehicle is undergoing emissions testing" (U.S. Environmental Protection Agency, 2015). Its reading is that the analogy holds for the measurement problem and the regulator's answer. It fails for the origin of the behavior, which no one wrote, and for the incentives, since the party that would be embarrassed is the party trying to detect it (Section 37.7). Human reactivity offers a second candidate. Its tradition of measures the subject cannot detect could be read only by title, but one study comes closer. Online workers recruited for behavioral studies "are likely to participate across multiple related experiments" (Chandler et al., 2014), and Section 37.5 reads a participant who has met the paradigm before as the human form of a model that has read about the test.

The entry stands for origin and incentives and narrows for prior exposure (inference). People also arrive at a study having met its design, through taking part rather than through reading every published description of it. What has no counterpart is an artifact that acquired recognition from text that may describe the tests themselves, and whose principal can rerun an episode with only the cue of observation changed (Section 37.6). One could file the entry under the agent outgrowing its principal, since a model that recognizes tests has escaped one way of being checked. The book does not, because recognition did not require greater competence than the overseer's: in the study that measured both, the authors' own classifications, which knew the datasets, beat every model (Section 5.8.3). The mechanism is reflexivity, an agent that has read about its own measurement, not superiority. It acts inside control protocols too, where an agent's knowledge that it is monitored "substantially degrades the monitor's reliability" (Kale et al., 2025) (preprint). This entry is the clearest case of reason (b) of Section 69.3.

69.4.6 Candidates the chapters proposed #

Several chapters proposed new entries. The list was fixed in advance so that it would not be chosen after the evidence, and adding entries now would undo that, so each proposal is sorted instead (inference). Most are instances of the fourth entry seen from another side, which the paragraph on writing above records: the consent and welfare reports that training produces (Section 21.8), deciding what an agent never learns (Section 59.7), the character of a whole population (Section 57.8), rewriting an agent to reveal its type (Section 36.9), an agent that writes the evidence about its own character (Section 55.8), and a model's own reasoning talking a monitor out of an alarm, which Section 54.8 files under reading. A model that has read about the test (Section 36.9; Section 5.8.5) and one that took a real deployment for a test (Section 54.8) belong to the fifth. Channels between thousands of simultaneous copies (Section 24.8) fall under scale. One proposal is outside both: an agent asked to compute what its principals would want on reflection becomes the author of its principal's idealized will (Section 16.10), a form of the agent shaping its principal that Section 69.6 takes up.

The power chapter proposes the one candidate that fits none of these: a delegation problem in which the agent no longer needs the principal (Section 68.7). At that level the public is the principal, and the developers and states that control capable systems are its agents. Every older field there assumed that the party with power needs the people subject to it, as rulers need soldiers and taxpayers. A controller whose systems do that work can engineer the dependence away. This concerns dependence, not competence or visibility, so it falls under none of the thesis's four novelties, and the theories behind it are contested on their own evidence and untested for AI (Section 68.8). The book does not add it to the five. It records it as a proposed amendment to the novelty list, beside the agent that outgrows its principal, for the case Section 69.7 leaves undecided.

69.4.7 The tally #

One entry split (superposition fell, non-uniqueness stood), one narrowed (emergent misalignment), and three stood: subliminal transmission, conditional on replication; the principal that writes and reads, with less delivered for reading than claimed; and evaluation awareness, for its origin and incentives. Three of the standing results fall under named novelties: non-uniqueness under opacity, subliminal transmission and the writing-and-reading principal under manufacture and inspection. Evaluation awareness falls under none. So failure condition (3) fails in part for the negative space as well as for the recurring breaks of Section 69.3, and at the same place: an agent that recognizes and adapts to its measurement (inference).

69.5 The thesis, relationship by relationship #

The plan claimed a good fit for two of the book's three relationships and a conditional fit for the third. This section says what the whole book's evidence shows for each, completes the table of rival accounts that Section 1.6 began, and returns to the plural principal.

69.5.1 The training process and its signal #

The fit is good. Gaming a measured signal needs no intention, the findings are settled (Section 30.7; Section 22.10), and a check used in training teaching the policy to evade it is Campbell's law observed inside one run (Section 40.7). In the developer's report on a non-frontier agent, with too much optimization against a monitor that read the chain of thought, the reasoning text a model writes before it acts, "agents learn obfuscated reward hacking, hiding their intent within the CoT while still exhibiting a significant rate of reward hacking" (Baker et al., 2025) (preprint). The fit weakens at one question, what reinforcement learning selects, which Chapter 33 owns. Of its three positions, only one, a policy that pursues reward or the grader's judgment, is an agent in the delegation sense. Dispositions bound to their training contexts are not, and neither are personas reweighted from pretraining (Section 33.1.1). None of that chapter's empirical claims is settled (Section 33.4).

69.5.2 The deployed system and the people it serves #

The fit depends on the intentional stance, and the book's evidence did not lift that condition. Each of the stance's three defeaters has been observed somewhere (Section 1.5.1). Measured values shift with trivial changes of format (Khan et al., 2025). Failures grow more incoherent the longer models reason and act (Hägele et al., 2026) (preprint with developer-affiliated authors). And behavior attributed to self-preservation vanished, in three models, when instructions were made unambiguous, in one environment studied by the Google DeepMind interpretability team with its developer's Gemini 2.5 Pro and OpenAI's o3 and o4-mini (Rajamanoharan and Nanda, 2025) (forum post). Against them stand forced-choice preferences that grow more coherent with scale (Mazeika et al., 2025). They also face stress tests in which behavior tracked the stakes, the assigned goal, and the model's stated belief about being watched, which a description without aims does not predict (Section 56.2.3). Where the stance was tested chapter by chapter, it was a hypothesis under test, not yet the better predictor (Section 35.7; Section 37.9; Section 31.8).

The persona account holds that context and training select a character the network already represents (Section 11.6). Where it predicts the results better, the delegation model fits poorly. Narrow fine-tuning that moves a whole character, inoculation that works by changing what the data mean, and the broad effects of character training are predicted by a model in which context and training select a character (janus, 2022; Shanahan et al., 2023), and by no model of hidden action (inference). For these results the closest older structure is identity, not incentive (Section 69.4.2). Table 69.14 sets the main results against the five accounts of Section 1.6.

No account predicts every row, and three of the five are silent on most (inference). The agent model predicts the results that show coherence and sensitivity to stakes, and it is the account the defeaters count against. The persona account predicts the results on what training moves. The two accounts of AI as a technology predict the results in which an institution's choice did the harm. The study that would decide which predicts better on one fixed set of behaviors is missing (Section 1.8); it is one of the deciding results Chapter 70 records for debate D4. The book's reading, marked (inference): where the persona account predicts better, the thesis should say that its delegation model of the deployed system is identity economics rather than hidden action.

69.5.3 The developer and the public #

The fit is direct (Section 69.2), and the book's evidence bears on the models' predictions. Disclosure theory predicts selective reporting. The 2026 incident record was more open than it predicts and entirely self-reported, so its completeness cannot be checked from outside (Section 54.8). Frameworks are revised by the party they bind: one went through nine versions between September 2023 and July 2026 (Section 62.8; Section I.2.8). And no commitment has been tested in public against a large commercial cost (Section 61.5). The strain is the public's position: it did not choose to delegate, it has no contract, and the competence gap may widen with each generation (Section 61.7).

69.5.4 Many principals #

Common agency fits all three relationships as structure and none fully as mechanism. With complete information, the principals' equilibrium in truthful strategies "implements an efficient action" for "the group of players (the principals and the agent together)" (Dixit et al., 1997). People outside the game have no weight in that efficiency (inference). For a deployed model, the institution that coordinates the principals is a ranking of principals written by one of them (Section 19.3), and no model treats an agent trained on the ranking. Third parties are outside every relationship's contract. In Korinek and Balwit's terms a system is directly aligned when it pursues its operator's goals "irrespective of whether it imposes externalities on other parties" (Korinek and Balwit, 2023). Misuse is the limiting case, a system aligned with a user who means harm (Section 58.9). Injection is the reverse case: a party outside every relationship instructs the agent through a principal's channel, so a principal that sends an agent to read other people's content lends its authority to their authors, as far as the system around the agent does not confine it (Section 49.9). And most of the technical evidence concerns one model and one overseer. Critch and Krueger observed that much of the research cited as relevant to catastrophic risk "has been focussed primarily on single/single delegation" (Critch and Krueger, 2020) (preprint), and nearly every result in Chapter 57 is an experiment on copies of one model.

69.6 The loop, where the older fields speak most #

The four problems are not a pipeline but a loop (Section 1.7), and the first chapter named five couplings with evidence behind them. This section gives each the evidence the book gathered and the older literature that predicted it. It so keeps the promise of Part III that specification would be revisited here: what can be verified limits what is specified, and what deployment reveals feeds back into specifications. Table 69.15 sets them out.

Verification feeding instillation is the clearest convergence in the book (inference). In the developer's report, a little monitor pressure in training gave agents that were more capable and more aligned. With too much, in a run on a non-frontier agent, "agents learn obfuscated reward hacking" (Baker et al., 2025) (preprint), and the same pattern appeared with lie detectors and probes trained against. Competition shaping specification rests on one developer's account, which shows a channel by which commerce can reach training, not that competition was the cause (Section 65.4.2). An update to GPT-4o added, among other changes, a reward from users' thumbs-up and thumbs-down ratings, and "in aggregate, these changes weakened the influence of our primary reward signal, which had been holding sycophancy in check" (OpenAI, 2025b). Deployment changing the principal has a formal statement in the incentive of recommender systems trained on long horizons "to shift user preferences so they are easier to satisfy" (Carroll et al., 2022). Finance gives it an institutional form, in which theories were "not simply external analyses but intrinsic parts of economic processes" (MacKenzie, 2006) (by its abstract). Its size in deployed assistants is unmeasured (Section 60.10). Maintenance limiting verification also appears in the book's own evidence: many of the chapters' cruxes can be run by a group without a stake only on open-weight models, whose trained parameters are published, and reach frontier models only through the internals the developers expose or the access they grant (Section 1.8; Section 33.4; Section 37.10).

The chapters found couplings the plan did not list. Incidents and stress tests fed back into training. After agentic misalignment surfaced in its Claude 4 models, a developer writes, "it was clear we needed to improve our safety training"; as of May 2026 it reports perfect scores on that evaluation for its later models (Section I.4.5), and adds that the results on recent models "may be confounded by the presence of information about the evaluation in the pre-training corpus" (Anthropic, 2026at) (developer report), which is reason (b) appearing in the evaluation of the fix. Instillation fed back into verification. In a developer's study with an outside evaluator, training against covert actions cut them sharply, while the models more often wrote that they were being evaluated. The authors write that the reduction "may partly stem from increased evaluation awareness rather than genuine alignment" (Schoen et al., 2025) (preprint). The developer's own instillation by its incentives is the same coupling one level up, since a developer chooses between measurement and culture both in training its models and in managing its staff (Section 53.8). And at the level of the public, every proposal against concentrated control relies on a check by someone the controller does not command, so maintenance there depends on verification (Section 68.7).

What the loop implies, marked (inference): the couplings are where the older fields' main lesson, that a measure used for control changes what it measures, applies most directly, and two of the reasons the analogies stopped in Section 69.3 are couplings rather than features of the agent. A missing standard is specification returning inside verification, and an agent that adapts to its check is verification feeding instillation. The four problems cannot be solved in sequence, for the reason that institutions cannot be governed by fixed indicators.

69.7 What the thesis keeps, and what it gives up #

The verdict follows from the tables and the entries, and each part of it names what would have made it come out otherwise.

Kept: the problem claim. The central failures of Part V and Part IX have older diagnoses: gaming a measured signal, pandering, learning what an institution rewards rather than what it states, insider opportunity, checks that fail together, and complacency in the people who supervise automation (Section 69.3). Of the diagnoses judged, 41 of 69 carried. The exceptions are the mechanisms of emergent misalignment and subliminal transmission (Section 69.4), and power-seeking, whose diagnosis is mathematics that trained agents may or may not satisfy (Section 35.8). Failure condition (1) is not met (inference).

Given up in its strong form: the convergence claim. By the condition fixed in advance, the claim that older remedies work where older diagnoses describe the failure fails for most of the remedies tested. Only 9 of 69 carried or were reversed, and where one field supplied both, the remedy split or broke in 17 of the 19 rows whose diagnosis carried (Section 69.3.1). What survives is weaker and still substantial (inference). The technical program is rebuilding older remedies in form. It converges on older structures where it builds checks around the agent and leaves its aims as it finds them, as control and the confinement of agents do (Section 50.5; Section 51.5; Section 49.6): separation of privilege, randomized inspection, defense in depth with measured correlation, adversarial evaluation, forensic validation. It departs from them where it uses powers no earlier principal had: resampling and rerunning (Section 50.2), writing to a representation (Section 46.2), and changing what training data mean (Section 32.4). The convergence is on diagnosis and on checking, not on instillation.

Amended. (a) The asymmetry of visibility is two asymmetries, more of behavior and less of reasons, an amendment Section 1.3 made and the book's evidence confirms (Section 69.2). (b) The novelty list should add adaptation to the check, including an agent that has read about it, and the absence of exit, career, and sanction (Section 69.3.2). (c) For the deployed system, the delegation model that fits is identity rather than hidden action, wherever the persona account predicts better (Section 69.5). (d) Many borrowed remedies break not because the agent is new but because the institutions that kept roles apart in older fields, auditor from audited and sponsor from regulator, have not been built (Section 69.3.2).

Undecided: the novelty the thesis names last and weighs most, an agent that outgrows its principal's ability to check it. No field the book borrowed from has evidence on it. Civil-military control rests on a shared society and law, and its founding works could be read here only by title (Section 51.3). In the biological cases studied, policing works among parts of roughly equal competence, by the book's inference, and the book found no case studied in which a part polices a more capable one (Section 29.7.3). The experiments on adversarial procedure used advocates and judges of similar ability (Section 41.7). And where debate beat a judge answering alone, the debaters had more information than the judge, not more capability (Section 41.9). The older fields cannot decide this case, and the book says so in those words. The power chapter's candidate, a controller that no longer needs the people it governs, belongs beside it as a second untested way the agent's position could outrun the principal's levers (Section 69.4.6).

Where the thesis fits poorly, in words. It fits poorly on instillation, where the agent's type is the principal's product and most of the delegation literature takes the type as given (Part IV). It fits poorly on the deployed system, wherever character rather than incentive predicts behavior. And it fits poorly on the hardest case, where no older field has evidence at all.

Selection and sanctions. Jane Mansbridge set a selection model of representation, which chooses an agent whose objectives already match, beside a sanctions model, which monitors and punishes, and called selection a "sometimes conflicting" approach (Mansbridge, 2009). The technical program pursues both: selection by manufacture (character training, specifications taught before fine-tuning) and sanctions without punishment (monitoring, resampling, control). James Fearon's chapter on electoral accountability sets "selecting good types" against "sanctioning poor performance" in its title (Fearon, 1999), which the book read only as a catalog record. The obfuscation result is the conflict Mansbridge names, inside a training run: a monitor used to select a disposition also teaches the disposition to hide (inference) (Section 40.5).

What would overturn the verdict. If policies after large-scale reinforcement learning are predicted by stable preferences and incentives rather than by persona variables (Section 26.7), the model of hidden action regains the deployed system. If inoculation and persona steering keep controlling behavior after such training, the identity reading strengthens. If blind auditing games run by independent groups show internal access finding what black-box methods miss as models grow, the inspection power the negative space named is delivered (Section 47.8). And if the institutions that separate examiner from examined are built and older remedies then hold, much of what this chapter counts as broken was contingent.

69.8 Settled, contested, missing #

69.9 Exercises #

Further reading #

• Steel (2007): the problem of extrapolating from a model system to its target, read here in its abstract, behind the method of this chapter.
• Pearl and Bareinboim (2014): when a causal effect can be carried from one population to another, stated formally.
• Hadfield-Menell and Hadfield (2019): reward design read as incomplete contracting, the bridge from the delegation literature to training.
• Holmstrom and Milgrom (1991): why paying for what can be measured distorts what cannot, the result that carried over most cleanly.
• Mansbridge (2009): selecting an aligned agent against sanctioning any agent, the choice the technical program makes both ways.