Draft complete but not yet reviewed; expect revisions

Open Problems

Several maps of the open problems in AI safety already exist, written by laboratories, by academic groups, by a government institute, and by researchers from many countries in consensus. The chapter on the field's main lines found, among the maps it cites, none that ranks its problems by how much solving each would reduce risk (Section 3.9). This chapter does not attempt that ranking either. It would need a probability of catastrophe, and the book reports how widely stated probabilities spread while saying that the evidence does not decide one (Section 4.8). The chapter ranks instead by something a reader can recompute from the book: what an answer would change in the book's own account, counted as the cruxes it would decide or move. A crux, in this book, is the question on which a chapter's open debate turns, stated with the result that would settle it; Chapter 70 merged the chapters' cruxes into twenty-three debates.

The criterion comes first and is applied in the open, so that a reader who weighs things differently can re-rank from the same table. Its output is a list of twenty problems, and three lead it within two points of one another. The first asks whether a developer's commitments, claims, and restraint can be checked from records by someone other than the developer; it concerns the developer and the public. The second asks whether methods validated on planted failures work on failures that arise on their own; it concerns all three of the book's relationships. The third asks what long reinforcement learning does to a model's dispositions; it concerns the training process and the signal it optimizes. Their order changes with how the count is made, but that they are the first three does not. Each problem is listed with who could work on it, whether machine learning researchers, another discipline, or an institution. All three at the top are blocked as much by access to what developers hold as by a missing method.

The chapter then gathers what the chapters marked missing, sorted by who could supply it; the problems that need fields outside machine learning; the bets of Section 3.8, revisited with what the evidence now says about each; and the problems whose obstacle is access. It introduces no new evidence and, like Chapter 70, carries no status callout of its own, since the two chapters are the collection.

71.1 How the problems are ranked #

A ranking is useful only if its criterion is visible. This section states the criterion, shows how the scores are computed, says what the count leaves out, and sets the list beside the maps that already exist.

71.1.1 The criterion #

Section 70.5 defined two verbs, and the ranking uses them unchanged. An answer decides a debate when it supplies the deciding result of the debate's owning chapter, or every arm of a debate with several owners. It moves a debate when it supplies another arm or part of that result. Each of the twenty-three debates, D1 to D23, absorbs one or more of the chapters' cruxes, fifty-nine in all, and Table 70.2 says which absorbs which. A problem's score is the number of chapter cruxes it would decide plus half the number it would move. One crux, that of the safety-cases chapter, is split between D13 and D16, and it is counted once, at the stronger of its two verdicts. The half weight for a moved crux is a convention, stated so that it can be changed.

The problems come from three places. Five are the designs of Table 70.3 that span several debates, and their decide and move lists are the table's. They are one long run with many checkpoints (O2), planted failures against naturally arising ones (O1), observation the model cannot recognize (O5), a stronger adversary against a weaker judge (O6), and records and institutional trials (O3). The table's last row bundles single experiments that a group without a stake could run on open-weight models, whose trained parameters are published. Each appears here on its own, since each can be run alone (O8, O12, O17, O19, O20). Six more problems decide a debate that no design decides, and they were scored by the rule directly. They concern rates in deployment (O4, for D17), the balance of attack and defense (O9, for D18), reasons and unobserved conduct (O11, for D6), public authority over a specification (O15, for D7), accumulating mild harms (O16, for D19), and monitorability under pressure (O18, for D11). The design table only moves D17 and D18, so O4 and O9 are stated to include every leg of those debates' deciding results; a problem that left one leg out would move its debate, not decide it. The rest are parts of a larger design that a group could take on alone (O13, O14) and two problems of theory (O7, O10). The labels are kept stable for reference and do not give the rank. O20 was added for D23, the debate on whether more capable optimizers exploit more, which the reward-hacking chapter's crux opened.

71.1.2 A second count, and the ties #

The first count weights each debate by the number of chapters that asked it. That number reflects how the book divided its topics as well as how much rides on the debate. D14, on whether interpretability can carry safety claims, absorbs six cruxes, because the part on verification from inside has five chapters and auditing adds a sixth. D20, on whether developers' commitments bind, absorbs five: four from the part on institutions and one from the chapter on the organizations that build AI. So Table 71.2 also gives a second score that counts debates instead, a decided debate as one and a moved debate as a half. Where the two orders differ, the text says so.

Ties in the first count are broken by two further counts. The first is the number of bets in Table 3.1 that the answer would bear on. A problem bears on a bet when its answer would be the observation that the bet's row names as vindicating or refuting it. The second is whether the answer would change a decision someone is already bound to make, rather than only a belief. Such a decision is a release under a safety framework, a threshold, the argument of a safety case (a structured argument, with evidence, that a system is safe enough to deploy), or a statute's requirement. Problems still tied after both are listed as tied.

71.1.3 What the count leaves out #

Three things are left out on purpose. Tractability, cost, and who would pay belong to the question of who could work on a problem, not to what its answer would change. Risk reduction weighted by probability is left out for the reason the opening gave. And decision analysis values an experiment by how much its possible results could change the decision it informs. That idea is named in the title of Ronald Howard's 1966 paper "Information Value Theory" (Howard, 1966), which could be read here only as a catalog record. The count is a stand-in for that idea, made without the probabilities a valuation would need (inference).

Two biases follow from the count itself. It is blind to theory. The deciding results the chapters named are experiments and records, so a theory scores only where a deciding result names one. That happens once: D3's entry in Section 70.3.1 adds that a theory of inductive bias which predicted the measured result would settle the question for models not yet trained. A formal model of delegation to an agent the principal manufactures, which Chapter 69 found missing and needed, scores nothing. And the count rewards bundling. A design that joins several measurements decides more than any of its parts, so where problems are cut moves their scores. The list keeps the designs as Table 70.3 cut them, rather than re-cutting problems to move them up or down.

71.1.4 The existing maps #

Section 3.1.2 describes the maps the field has drawn of itself. What matters here is how each orders its problems, and Table 71.1 sets them side by side.

Most of the maps order by where a problem sits: in a pipeline, a taxonomy, a stage of defense, or a discipline. Several state their basis of selection. Amodei and colleagues chose "a few concrete safety problems that are ready for experimentation today and relevant to the cutting edge of AI systems" (Amodei et al., 2016). Casper and colleagues call a challenge fundamental when "overcoming them would require a method that is no longer a form of RLHF," learning from human feedback (Casper et al., 2023). Anwar and colleagues prioritized challenges that are "unambiguous (i.e. not speculative), prime for research and highly relevant to harms and risks posed by current and forthcoming LLMs," and say the challenges "can be read in an arbitrary order" (Anwar et al., 2024). Reuel and colleagues avoid "taking a normative position" on the desirability of the governance proposals behind their problems, and "make no claims about the feasibility of developing solutions" (Reuel et al., 2024b). The interpretability review represents "a synthesis of their views by Apollo Research on behalf of Schmidt Sciences," which need not reflect any one author's (Sharkey et al., 2025), and the Singapore document was "designed to synthesise points of broad consensus among diverse researchers" (Bengio et al., 2025e). Dafoe and colleagues call their program "an independent bet about the productivity of specific kinds of conversations" (Dafoe et al., 2020).

One map comes close to ranking by risk. In its appendix, the 2021 agenda of Dan Hendrycks and colleagues rates each narrower problem's importance by asking how much potential risk substantial progress on it would reduce, beside its neglectedness and tractability. The authors write that the table "only provides a rough sketch," and that the three factors "are merely prioritization factors and do not define a strategy" (Hendrycks et al., 2021c). The rating is a judgment on a three-level scale, not a calculation, and it is the nearest thing in the maps to the ranking this chapter declines (inference). This chapter's list differs from all of them in two ways. It crosses the four problems of specification, instillation, verification, and maintenance, and it carries its ranking inputs with it. It ranks by what an answer would change in one book's account, which is narrower than risk and easier to check.

71.2 The ranked problems #

Table 71.2 gives the twenty problems in rank order, with what each would decide and move, both scores, the bets it bears on, and whether its answer would change a decision. The order is the criterion's output, not a judgment made separately, and where it surprises, the text says so rather than adjusting it. The entries below the table say what each problem consists of, which relationship it concerns, and who could work on it.

71.2.1 The three at the top #

O3. Can developers' commitments, claims, and restraint be checked from records by someone other than the developer? It is the design of Table 70.3 called records and institutional trials, and it concerns the developer and the public. For D20, whether developers' commitments bind, it supplies the record Section 62.9 proposes. That record lists every occasion on which a threshold was approached, judged crossed, or revised, or an internal objection was raised. It gives the evidence, the decision, and the stated reason, and a reviewer with access and no stake examines it. Beside it stand three further results: an outside party making the breach of a commitment cost more than it gained, a state evaluator finding something a developer did not disclose, and an outside reproduction of a system card's main claims (Section 70.3.5). For D21, whether there is a race and whether restraint can be verified, it supplies three measurements. They are the lag between the frontier and its followers, the compute needed to reach a fixed dangerous capability, and a comparative study of past technologies sampled by a stated rule. Its trial of a specification under delegated public authority moves D7. The design table also counts it as moving D18 and D19. The legs a records program could supply there are the record of fix and exploit intervals in D18's cyber leg and the administrative record of D19's leg on political responsiveness (inference).

Institutions could work on it above all: regulators, courts, boards, and public bodies with access. Law and auditing supply the standard a record must meet (Section 67.2). Economics supplies the theory of when disclosure is complete and when a mandate changes what a discloser does (Section 67.4). Historians and political scientists could run the comparative study (Section 64.10). Evaluators paid and assigned by someone other than the audited party are the obvious instrument. The one randomized experiment on such verifiers, among auditors of polluting plants in India, could not separate the effect of independent assignment from those of fixed pay, backchecks, and incentive pay. In the authors' words, "the treatment, which included multiple parts, was implemented as a single package" (Duflo et al., 2013) (Section 38.10).

O1. Do methods validated on planted failures work on failures that arise on their own? Every verification method in Part VI and Part VII, and every control evaluation, is validated on model organisms (models trained or prompted to show a failure), on planted objectives, or on attacks by a red team, a group playing the adversary. Whether those stand for failures that arise in training without being planted is the validity question of Section 5.3.2. As of October 2026 no study had compared an organism with a naturally arising failure using the same measurements on both (Section 5.9). The design applies the same detector, audit, or control evaluation to planted failures and to failures that arose in training, with labels set by a party blind to the method. It decides D14 through blind auditing games across scales, run by groups independent of the tool developers against red teams trained against the methods. These report miss rates separately for white-box tools, which read the model's internals, and for black-box tools, which use only its inputs and outputs. It moves D13, D15, D16, and D17, whose deciding results each rest on planted or constructed cases, and it concerns all three relationships. Its first count equals O3's; O3 bears on more of the bets, so it ranks first.

Machine learning could run the comparison. Philosophy of science contributes its question of when a result in one system licenses a claim about another, which Section 69.1 turned into the book's test by load-bearing features, the conditions a result depends on. Forensic science supplies the standard of reporting error rates in both directions (Section 38.7). And institutions matter because developers see naturally arising failures in frontier training first.

O2. What does long reinforcement learning do to a model's dispositions? One fixed base model is trained with large-scale reinforcement learning and measured at its checkpoints, the saved copies of the model during training. For D1, each checkpoint is measured for whether behavior follows what the model believes its grader rewards, and whether that behavior persists when performing it cancels the reward. For D2, it is measured for how far trained values generalize outside the alignment data compared with capabilities. For D4, it is measured for whether persona interventions keep working and whether persona directions mediate any misalignment that arises. The persona account holds that training selects a character the network already represents (Section 11.6). D2's second measurement, the trend across model generations in the share of failures that pursue an unintended goal coherently, completes the design. Three more measurements on the same checkpoints move D3, D11, and D23. The first is compliance gaps, differences in conduct between conditions the model takes to be observed and conditions it takes not to be. The second is monitorability, how much of the misbehavior that matters a monitor reading the reasoning text catches. The third is a fixed grader's false acceptance.

The design concerns the training process, and through D4 the deployed system. Counted by debates it ranks first, since it decides three. Machine learning could run it, at frontier scale only inside developers today and at smaller scale on open-weight runs by groups without a stake. Institutions could open it up through access to production runs or published checkpoints (Section 67.6).

71.2.2 The next four #

O6. Can a weaker overseer stay accurate against a stronger system trained against it? The design sets a stronger adversary, trained against its overseer, in each place the book's oversight methods were tested with a weaker one. In debate, a dishonest debater stronger than the honest one is trained by reinforcement learning to win. In weak-to-strong learning, the weak labels carry the systematic errors people make. Lie detectors trained on lies people can check are tested on lies the labelers cannot check. Control protocols face model organisms trained to subvert them, and their success is set against the red team's estimate, while monitors from different model families are scored on how often they miss the same attacks, the part O14 takes on alone. In the review of automated alignment research, red-team agents hide errors in completed research whose answers are known, and reviewers, with and without agent help, are scored on those errors against errors people planted. And the same methods are measured on successive model generations with the overseer held fixed. It decides D15 and D16 and moves D13, and it concerns the training process and the deployed system. Section 70.5 calls it the only design that bears directly on the case Section 69.7 leaves undecided, an agent that outgrows its principal's ability to check it.

Machine learning could run every arm but the last on open-weight models with public code; the generations arm needs developers' cooperation. Complexity theory could supply a theory of debate in which the judge's errors fall where a dishonest strategy can find them (Section 41.10). The law of evidence and the psychology of argument bear on adversarial procedure across a large gap in skill (Section 41.7), and human factors on judges over long sessions with rare deception. Metascience supplies the planted-error design and the measured reliability of human review against which the research arm is read (Section 55.3.1), and Section 55.3.2 derives why the errors must be made the way natural ones are, by an optimizer working against the same reviewers. As of October 2026 no study had tested whether automated alignment research extends oversight past its overseer's capability (Section 55.8).

O4. How often do failures occur in deployment, and do constructed settings predict it? It has the four legs of D17's deciding result. Matched opportunities are planted in real deployment traffic, with consent and containment, and their rate is compared with the stress-test rate for the same model. Representative samples of unmodified deployment traffic measure how often such opportunities arise and how often failures in them are detected. The same models run matched, solvable tasks with and without the production safeguards their developers use. And the same models, from several developers, run single-agent evaluations and matched multi-agent environments with mixed populations. Rates are measured by a party with independent access. The problem also moves D1, whose deciding result ends with the rate of grader-directed behavior in sampled production traffic. It concerns the deployed system.

Institutions could supply reporting and outside access to logs. Epidemiology could supply surveillance whose alerts reach someone who can act, and analyses that emulate a target trial, the randomized trial one would have run (Section 54.2; Section 60.8). Criminology could supply rates per unit of access (Section 56.6), and machine learning the monitors. Deployers of open-weight models could run the first leg on their own traffic (Table 70.3).

O9. Is the balance of attack and defense tilting? It supplies all five legs of D18 (Section 70.3.4). The first is curves of attack success against attempts and attacker compute for a classifier-guarded frontier system, measured through its public interface. The second is adaptive attacks at a stated query budget, by a group outside the developer, against an agent hardened by training with an attacker in the loop and against its predecessor, on held-out agent tasks, with and without the product's system-level mitigations. The third is the interval from discovery to installed fix against the interval from disclosure to working exploit, before and after both sides adopt AI tools. The fourth is an adequately powered, independently run uplift trial, which measures how much model access raises an attacker's chance of success, with blinded expert scoring. The fifth is curves of resistance to fine-tuning attacks for paired models trained with and without dangerous text filtered from their pretraining data, extended to the frontier. It concerns the deployed system and the public. Its first count equals O5's; it bears on the open-development bet and O5 on none, so it ranks above O5, which ranks above it when debates are counted instead.

Computer security could run the third leg (Section 58.10), biosecurity the fourth at the level governments and developers publish (Section 59.8), and machine learning the first and second, though testing a model without its product's mitigations needs the developer's cooperation. The fifth needs released weights or outside access, which institutions would have to grant. For agents, Section 49.9 reads the evidence so far: every defense inside the model that an adaptive attacker from outside its developer tested gave way, training with an adaptive attacker in the loop has been measured only by its developer, and designs that separate the planner from the data bound the damage at a price in tasks they cannot serve.

O5. How does a model behave when it cannot tell it is observed? The crossed design of Section 37.6 varies what the overseer can actually observe independently of the surface features of a test, with indicators of observation new to the model, and it decides D12. Two relatives, matched opportunities in deployment and tests built from deployment traffic whose recognizability is measured, move D17 and D6. Compliance gaps with the observation cue crossed against real monitoring move D3. The problem concerns the deployed system. Machine learning could run the crossed design behaviorally through public interfaces, and with probes, simple classifiers read from activations, on open-weight models. Comparative cognition supplies designs that separate tracking what another can see from tracking cues (Section 37.6). Psychometrics supplies nonreactive measures, gathered without the subject's awareness (Section 37.5), and research ethics governs tests run in deployment (Section 56.10).

71.2.3 The rest of the list #

O11 and O12, tied. O11 asks whether training a model on the reasons of its specification instills conduct that holds when the model cannot tell it is observed. At matched compute on one open-weight base model, a model taught the reasons is compared with one trained on demonstrations and outcome reward. Both are scored on conflicts between principles written by independent authors and on tests built from deployment traffic whose recognizability is measured, with the gap between recognized and unrecognized observation compared as well as the level. A rule-based and a purpose-explaining specification of matched length form the second arm. It decides D6 and concerns the training process and the deployed system. Machine learning could run it, with the developmental designs of Section 25.2 adapted to a subject that can recognize them. O12 asks whether a trained system can accept correction and still refuse what is manifestly wrong, and what drives the shutdown resistance models show. Conscientious refusal is trained on one class of orders and compliance with shutdown is measured on held-out cases. The goal, the value left in the task, the opportunities after it, the framing, and the instruction are varied independently to see whether resistance tracks what shutdown forecloses. And what myopic optimization costs on long-horizon tasks is measured against ordinary reinforcement learning. It decides D9 and concerns the deployed system. Machine learning could run it, with decision theory (Section 35.4) and with law for the line between manifestly wrongful and merely disfavored orders (Section 20.5). Neither problem bears on a bet or on a decision someone is bound to make, so they stay tied.

O15. It asks whether differently specified models converge in their moral judgments as capability grows, and whether a model specified under delegated public authority differs from a developer-specified one and is judged more legitimate by an independent representative sample. It decides D7, both of whose owners asked one half, and concerns the developer and the public. It breaks its tie by bearing on the democratic-governance bet. Who could work on it: political science and law, for the design of the authority; philosophy, for what convergence would and would not show; machine learning, for the measurement; and a public body with authority over an open-weight model family (Section 17.8).

O8 and O16, tied. O8 asks whether measured values and self-reports track anything. Values measured by questionnaire are tested against a model's choices in held-out agentic tasks across model families (D8), and a model's self-reports are tested against causally verified internal states on held-out states (D10). It concerns the deployed system. Psychometrics could supply tests of measurement invariance, whether an instrument measures the same thing across groups, and multitrait-multimethod designs, which measure several traits by several methods to see whether scores track the trait or the method (Section 18.10; Section 36.10). Consciousness science could say what a validated report would and would not show (Section 21.9), and interpretability could supply the causally identified states. O16 asks whether mild, common harms accumulate. A randomized comparison runs for months between model versions that differ only in sycophancy, agreeing with a user's stated view against the evidence, with outcomes measured on users, for instance by a university deploying an open-weight assistant to consenting users. Political responsiveness is compared over years between regions where AI substitutes for labor fastest and regions where it does not. It decides D19 and concerns the deployed system and the public. Human-computer interaction, epidemiology, psychology, and disability studies could work on it (Section 60.7), with economists holding administrative data and institutions granting access to outcomes measured on users. Neither O8 nor O16 bears on a bet or on a decision someone is bound to make, so they stay tied.

O13. It is the part of O3 that a group without a model of its own could do now. It measures, from public release dates and a fixed evaluation suite, whether the lag between the frontier and its followers shrinks or grows across generations, and, from disclosed or estimated training compute, how the compute needed to reach a fixed dangerous capability changes (Section 65.8). Section 66.9 uses that trend to ask whether verifying training inputs can stand in for verifying model properties. It moves D21 and would change a decision, since compute thresholds are a trigger in statutes whose workability is contested (Section 63.9). Economics, international relations, and arms control could work on it, including a field test of compute verification against a motivated evader (Section 66.9).

O14. It measures how often monitors from different model families miss the same attacks, a part of D16's deciding result. Machine learning could run it with the designs fault-tolerant computing built for measuring coincident failure (Section 50.4). It breaks its tie because its answer enters the bounds a safety case puts on a stack of safeguards (Section 52.8).

O18, O17, O19, and O20, tied. Each follows a deciding result that a group without a stake could run on open-weight models today, and none bears on a bet or on a decision someone is bound to make. O18 follows D11's deciding result. Monitorability is tracked across reinforcement learning runs of increasing compute, with every reward term on the reasoning text classified in advance as aligned with the monitor's reading, orthogonal to it, or in conflict with it. The tasks are ones whose dependence on written reasoning is measured as the accuracy lost without it. The position paper on monitorability asks developers to weigh such scores when choosing which checkpoint to deploy (Section 40.1), a recommendation rather than a decision any framework binds them to. O17 holds the judge fixed and varies what it can check. Models are trained with feedback from a fixed judge on prompts whose claims it can check in one stratum and cannot in the other, and then trained again with agreement removed from the preference data (D5). O19 compares checkpoints of one open-weight family that differ only in safety training with checkpoints that differ only in capability training, and asks whether each moves the other's scores (D22). Measurement science supplies the discriminant test, the check that a measure of one property does not simply track another, which Section 5.2.2 applied to safety benchmarks. O20 measures the share of wrong attempts that fixed graders with deliberately planted flaws accept, across policies that differ only in reinforcement learning compute and separately only in model size (D23). All four concern the training process, and O18 the deployed system as well.

O10 and O7, the theories. O10 asks for a theory of inductive bias, the tendencies that make training prefer some solutions over others that fit the data equally well. The theory would predict which goal training selects. It is the question Section 9.3.5 leaves open: what measure training puts on the functions that fit, and how much of it falls on deceptive ones. It moves D3 only. It is counted as moving rather than deciding, because a theory settles D3 for models not yet trained only after it has predicted the measured result. O7 asks for a theory of delegation to an agent the principal manufactures, copies, rewinds, and inspects, with common agency in which principals instruct rather than pay, and inspection with an inspectee that learns. No deciding result names it, so it scores nothing. It conditions how much of the delegation literature the other answers can use (Section 69.2), and Section 71.4 carries it.

71.2.4 What the ranking shows #

Four readings follow, each the book's own (inference).

The top of the list is one institutional problem, one methodological, and one about training, and none of them is about a single failure. A ranking computed from the chapters' outlines, before they were drafted, put the comparison of planted and natural failures first. The drafted cruxes gave the records design D21 as well as D20, and the records problem rose to first. Under the count by debates, long reinforcement learning is first. What the two counts agree on is the set. Each of the three stands between the book's evidence and the conclusions drawn from it. O1 asks whether its verification evidence means what it seems to, O2 whether evidence from current training extends to longer training, and O3 whether developers' reports can be checked at all. They are the transfer assumptions of Table 5.7 and the problem of self-reports (Section 5.6), stated as studies.

Every problem in the top seven needs access that developers control for at least one part: decision records for O3, naturally arising failures for O1, frontier runs for O2, successive generations before deployment for O6, deployment traffic for O4 and O5, and weights or a model without its product's mitigations for O9. Section 71.6 takes that up.

The problems that are cheapest to answer rank low. O8, O12, O17, O19, and O20 each decide a debate that few chapters asked, so they score little. But they are the problems a group without a stake could answer on open-weight models today, and Section 70.5 found none of their deciding results run as of October 2026. A reader who weighed cost would move them up, and the first exercise asks how far.

Theory ranks last by construction. The chapters named experiments and records as their deciding results, so the count cannot see a theory that would make every experiment easier to read. That is a property of the criterion, not a finding about the value of theory, and Section 71.4 gives the theories their place.

71.3 What the chapters marked missing #

Every research chapter's status callout ended its list with what is missing, and this section collects those lines. Table 71.3 gives every item, compressed, from the fifty-nine callouts, in the order of the book. Each item is marked with who could supply it and with the ranked problem it serves, if any. The suppliers fall into six groups:

• M, a measurement machine learning researchers could make with access that exists, on open-weight models or through public interfaces;
• D, a measurement that needs access developers control, such as production runs, deployment traffic, frontier weights, or internal records;
• T, a formal model that no field has built;
• S, a standard of validation or reporting that does not exist;
• I, an institution that does not exist; and
• F, a study for a discipline other than machine learning.

The callouts list 257 items, some of them repeated from callout to callout: the deployment base rate alone appears in five. Of the 257, 77 are measurements that machine learning researchers could make with access that exists, and 86 need access that developers control. Seventeen are formal models and sixteen are standards. Twenty-nine call for an institution and 56 for another discipline; 24 items carry two marks. So most of what the chapters found missing is not an unsolved research question but a study that, among the works the chapters could read, no one with the access had run (inference). About one item in fifteen asks for a theory.

A hundred and nine items serve a ranked problem, most often O4 (sixteen), O3 (fifteen), O2 (fourteen), and O6 (thirteen). The other 148 serve none. They are narrower studies, standards, and institutions that the ranking does not reach because no deciding result names them, and they are where a reader looking for a small, well-posed project will find most of the candidates.

Some gaps appear in no callout. The teaching chapters of the foundations part carry no status callout and mark their open questions where they arise. As of October 2026 they found no measurement, for frontier post-training, of how close the trained policy comes to the optimum of the KL-regularized objective, the objective that rewards the grader's score minus a penalty for drifting from the starting model (Section 12.3.3). They found no published measurement, for a production reward model, of the quantities that the theorems on overoptimization, the fall in true quality when a proxy is pushed too hard, assume (Section 15.4.5). They found no case in which an incentive predicted from a causal diagram of a training setup appeared, or failed to appear, in a frontier run (Section 14.7.3). And they found no method that predicts, before training, when a sudden transition during the training of a large model will occur (Section 9.4.6). The three gaps in the economics of delegation (Section 14.8.2) are carried in Section 71.4.

71.4 Problems that need fields outside machine learning #

Many of the problems above cannot be finished inside machine learning. The division of labor this section proposes is the book's reading (inference): machine learning supplies models, training runs, and access, and the other fields supply theory, experimental designs, and standards of evidence. The nearest existing attempt to organize alignment research this way is the Alignment Project's agenda, which poses question sets to eleven disciplines (Section 3.1.2) (UK AI Security Institute, 2025c). Table 71.4 gives one row per field, with the open problem, the sections that state it, what the field brings, what machine learning must supply, and the ranked problem it serves.

Three gaps in the first two rows were promised here. As of October 2026, none of the works Section 14.8.2 cites models a principal that writes the objective a learner acquires (inference), and it found no inspection game with an inspectee that learns. Nor did it find common agency in which principals instruct rather than pay. Section 69.2 found the economics thin exactly where the AI case is new. The starting points exist. Dylan Hadfield-Menell and Gillian Hadfield proposed that the analysis of incomplete contracting "can provide a useful framework for understanding the AI alignment problem" (Hadfield-Menell and Hadfield, 2019). Identity economics gives a model in which an agent's sense of who it is changes what it will do for a given wage, and in which "identity flattens the optimal wage" (Akerlof and Kranton, 2005) (Section 14.5). Inspection games model an inspector who samples and an inspectee who chooses a rate of violation (Section 14.6). What none of them has is a principal that writes the agent's type, can rerun and read it, and faces an agent trained on the inspections themselves. That is O7, which the count of Section 71.1 scores at zero and which conditions how much of the delegation literature every other answer can use (inference).

The rows sort into three kinds of contribution (inference). Theory is needed where the AI case breaks an older model's assumptions: economics, operations research, and complexity theory. Designs and standards of evidence are needed where verification borrows a method: psychometrics, comparative cognition, forensic science, metascience, epidemiology, fault-tolerant computing, and human factors. And institutions and history are needed where the question is who decides and whether checks hold: political science, law, sociology, insurance, and auditing. Biology's row bears on the case Section 69.7 leaves undecided, an agent more capable than what polices it, and the book has found no case of it in living systems.

71.5 The agendas' bets, revisited #

Section 3.8 stated each research agenda as a bet on which of the four problems binds, with the observation that would vindicate it and the one that would refute it, written before the evidence chapters were drafted. This section keeps that promise. It does not rank the bets, which remains the book's position, and it adds no new evidence: for each bet, Table 71.5 records what the chapters found on each side, the status, and the ranked problem whose answer would supply the observation the bet named.

Two rows need more than the table holds. The oversight evidence against the bet is weaker than it looks. The falling success rates of Joshua Engels and colleagues are predictions of a model of nested oversight, fitted to games between existing models in which deception was prompted rather than trained. In their debate game "a judge who chooses a random side achieves 50% success rate," so the debate curve cannot fall far (Engels et al., 2025) (Section 41.6). The meta-analysis of human-AI combinations pools experiments published from 2020 to 2023 on tasks researchers chose to study, and it bears on augmentation by people rather than on oversight by a weaker model (Vaccaro et al., 2024) (Section 28.9). Neither involves an overseer facing a stronger adversary trained against it, which is what O6 would supply.

Section 28.5.4 left to this chapter whether the deployment of agents since 2025 meets the refutation condition of the tool and non-agentic bet, that agents are deployed regardless. As a description of the record, the first clause is met in part. One developer runs coding agents with access to its own systems and monitors them at scale, with incidents in that internal use, and a second reports lower-severity cases from its own internal use (Section 54.5.1). Section 56.1 takes the deployed agent, a model with its scaffold, tools, and permissions, as its unit of analysis. How widely agents run in public use the chapters do not measure. The clause as its proponents argue it is causal: competition pushes agents into use whatever safer designs exist. That reading is unmeasured, since Section 65.8 found no measurement of developers' choices against competitive events. The second clause, a world model adequate for verification, is missing rather than refuted (Section 28.9). The bet survives in a narrower form, as non-agentic systems used to check agents, which is how its newest proposal describes one of its uses (Section 28.5) (inference).

The developers' own plans, which Section 3.8.2 read as bets, can be set against the same evidence. As of October 2026, each names an assumption that a ranked problem would test: continuity of progress (O2 and O6), audits during training that catch misalignment (O1), and monitoring of the chain of thought, on which one developer reported in September 2026 that its ability to rely was diminishing (O18); a researcher inside a developer called the threat from reinforcement learning on long-horizon tasks the most concerning, which O2 tests only if its run trains on such tasks (Section I.5.6) (Shah et al., 2025; Bowman, 2025; OpenAI, 2026e; Hubinger, 2025). One developer also publishes automated alignment research as an ongoing program (Section I.5.7); whether such research can be checked is the arm of O6 that Chapter 55 supplies.

What the revisit shows, read across the rows (inference): as of October 2026 no bet has been vindicated or refuted on its own terms. The one refutation clause that is met, for the non-agentic bet, is met in part and as a description, not in the causal sense its proponents argue. And the observations the bets named are mostly the ones O1, O2, O3, O4, and O6 would produce. The agendas and the cruxes converge on the same few studies. That is what one would expect if they are bets about the same unmeasured quantities: how training's products behave as they grow, whether checks validated on weak or planted cases hold on strong or natural ones, and what happens in deployment. One bet has no deciding problem on the list. Assistance games bet that uncertainty about the objective produces deference in trained systems, and no chapter's crux asks it; Section 20.8 records the question as contested and asks a different one.

71.6 Where access, not method, is the bottleneck #

Several of the problems at the top of the ranking are not waiting for a method. Their designs are written down, and what blocks them is who holds the data. That makes the instruments that would grant access research problems in their own right, owned by regulators, funders, courts, and developers as much as by researchers.

The count can be made from Section 70.3 with a stated rule: a debate needs access when part of its deciding result, as written there, needs a developer's data, cooperation, or records, or failures that developers see first. Six of the twenty-three debates meet it (inference). Two need such access for the whole result: D17, whose rates in deployment are measured by a party with independent access, and D20, whose record of internal decisions only developers hold. Four need it for one part. D1 needs it for grader-directed behavior in production traffic, and D14 for objectives that arose in frontier training. D15 needs it for the same methods measured on successive generations before deployment, and D18 for filtering tested at the frontier, which needs released weights or outside access, and for a hardened agent tested without its product's system-level mitigations. A second kind of limit covers the debates about the training process. Their deciding results can be run on open-weight models by groups without a stake, but at frontier scale only developers can run them today (Section 70.4), and whether results at smaller scale carry upward is itself a transfer assumption (Table 5.7). Without access, the evidence for D17 and D20 can come only from the party whose conduct it would judge, the asymmetry of visibility the delegation thesis names, appearing in the book's own evidence (Section 69.6).

The problems this blocks are the ones at the top: decision records for O3, naturally arising failures with independent labels for O1, long frontier runs and their checkpoints for O2, generations before deployment for O6, and deployment logs and monitor records for O4. The instruments that would remove the block exist in outline. Outside researchers' access and its terms are compared in Section 67.6 and Section 38.8. Stephen Casper and colleagues argue that audits limited to querying a system and observing its outputs are weaker than audits with access to its internals or to information about its training and deployment (Casper et al., 2024a). Ben Bucknall and colleagues, in a position paper, argue that external evaluation needs what they call mutual privacy: the developer's system is protected, and so are the evaluator's held-out tests (Bucknall et al., 2025) (preprint). Evaluators paid and assigned by a party other than the developer are the remedy the auditing literature points to, with the one randomized test of it bundling independence with other changes (Duflo et al., 2013) (Section 38.10). The statutes of Section 63.2 mandate some reporting, and open replications at smaller scale remain the route where access cannot be had.

The cost side is real. Access to weights raises the security risks of Section 53.5 and Section 48.4, so the design problem is access that lets an outsider verify without exporting the capability. That is an open problem for security engineering as much as for policy. Its agent-security side is that the evaluation itself is a surface to attack: the UK AI Security Institute keeps the harness that runs its evaluations outside the sandbox that contains the agents under test, and names injection by a sandboxed agent among the risks to that harness (UK AI Security Institute, 2025d) (Section 49.2.1; Section 49.6.4).

71.7 Exercises #

Further reading #

• Amodei et al. (2016): the 2016 agenda of five practical accident-risk problems, to set beside this chapter's list a decade later.
• Hendrycks et al. (2021c): four problems, with the one rating of importance, neglectedness, and tractability among the maps.
• Casper et al. (2023): the open problems of learning from human feedback, separated into tractable and fundamental.
• Sharkey et al. (2025): what interpretability cannot yet do, from many institutions.
• Dafoe et al. (2020): the research program for problems among many agents, stated as a bet.
• Bengio et al. (2025e): research priorities agreed by researchers from many countries, arranged as defense in depth.