Draft complete but not yet reviewed; expect revisions

Laboratory Safety Frameworks

A safety framework is a developer's written promise about itself: when its models reach stated levels of dangerous capability, it will apply stated safeguards, and if it cannot, it will not train or deploy them. The first was published in September 2023 (Anthropic, 2023a), and by December 2025 an evaluation organization counted twelve companies that had published one (Metr, 2025a). They are the most concrete answer developers have given to the question the previous chapter posed (Chapter 61): what the public's agent binds itself to, and how anyone would know whether it kept the promise.

This chapter reads the frameworks with the questions the book asks of a model specification (Section 19.2): who wrote the document, what it commits to, how compliance is checked, and who can change it. Its question is whether a commitment that its author writes, checks, and may revise constrains that author when keeping it is costly. The documents answer part of it directly. The mechanism they share, an evaluation that triggers safeguards, can be analyzed with the tools of Chapter 36, and its margin can be computed. The rest rests on a short revision record, on two fields that have studied rules firms write for themselves, and on a handful of outside reviews, and the chapter says where that evidence stops. Almost every claim here concerns the third of the book's relationships, the developer and the public; the trigger also governs what reaches the deployed system and the people it serves.

Several of the terms are the developers' own. "Responsible scaling" is one developer's name for its policy and "preparedness" another's. "Frontier" is the developers' and governments' word for the most capable models; a paper on thresholds notes that the term "has been accused of promoting a specific worldview" (Koessler et al., 2024), and a California statute now uses it as a legal category (Legislature, 2025b). The chapter uses "safety framework" as the neutral term and records the coinages in the glossary (Appendix A).

62.1 What a framework commits to #

What do the documents promise, in their own words? Every framework examined here has the same skeleton. A capability threshold names a level of dangerous capability at which new protections become necessary, such as meaningful help to someone trying to build a biological weapon. A safeguard is one of those protections. Security safeguards keep a model's weights, the trained parameters that are the model, from being stolen or altered; deployment safeguards keep users from drawing the dangerous capability out of a released model. Evaluations, test suites run against the model (Chapter 36), decide whether a threshold has been reached, and the framework says what must follow. This section sets out the first framework and the ways the others vary its skeleton, the international commitments that made publishing one a norm, and an outside count of what the published frameworks share.

62.1.1 The first framework #

Anthropic's Responsible Scaling Policy took effect on September 19, 2023 (developer document). Its central concept is the AI Safety Level, "modeled loosely after the US government's biosafety level (BSL) standards for handling of dangerous biological materials" (Anthropic, 2023a). Each level pairs a set of dangerous capabilities with two kinds of measure: containment measures, required to store the weights, and deployment measures, required to use the model inside or outside the company. The policy defined only the level its models were then at and the next one. It committed to define the evaluations for each further level before first training a model at the level below it (Anthropic, 2023a).

The policy states the consequence in one sentence. Following the scheme "implies that we commit to pause the scaling and/or delay the deployment of new models whenever our scaling ability outstrips our ability to comply with the safety procedures for the corresponding ASL" (Anthropic, 2023a). It scoped its concern to catastrophe, "for example, thousands of deaths or hundreds of billions of dollars in damage," and to two sources of it, misuse and autonomy (Anthropic, 2023a). Section 64.2 compares the scheme with the risk-tiered containment that biologists adopted in the 1970s.

62.1.2 Two more developers, and a laboratory's framework #

As of October 2026 the other reference frameworks keep the same skeleton and vary it in three ways; Section I.2.1 sets out each as written, with a table of all four. OpenAI's Preparedness Framework has two tiers of threshold, High and Critical, and its Critical thresholds "require safeguards even during the development of the covered system, irrespective of deployment plans" (OpenAI, 2025e). Google DeepMind's Frontier Safety Framework is built around Critical Capability Levels, in domains that include chemical, biological, radiological, and nuclear (CBRN) weapons and cyber attacks; for its misuse levels, the requirement that deployment mitigations bring the remaining risk to an acceptable level applies "only for external deployment, not internal deployment or further development" (DeepMind, 2026). A framework from the Shanghai Artificial Intelligence Laboratory with Concordia AI, written as guidance for developers, makes an outside review a condition of resuming after a model crosses one of its red lines (Shanghai AI Laboratory and AI, 2025). The variants are how many tiers of threshold there are, whether a crossing reaches development or only deployment, and whether anyone outside the developer must agree before work resumes.

62.1.3 The commitments that made it a norm #

At the AI Seoul Summit in May 2024, sixteen organizations agreed to the Frontier AI Safety Commitments (Department for Science and Technology, 2024; Metr, 2025a), and others signed later (Section I.5.3 gives the count as of December 2025). Each undertook to publish "a safety framework focused on severe risks" before the next summit, in France. The frameworks were to "set out thresholds at which severe risks posed by a model or system, unless adequately mitigated, would be deemed intolerable." The thresholds were to come with "an explanation of how thresholds were decided upon," and to be "defined with input from trusted actors, including organisations' respective home governments as appropriate." In the extreme, "organisations commit not to develop or deploy a model or system at all, if mitigations cannot be applied to keep risks below the thresholds" (Department for Science and Technology, 2024).

Two details of the commitments recur in this chapter. A footnote leaves the form of a threshold open: it can be defined using capabilities, risk estimates, safeguards, or deployment contexts, provided "it should be possible to assess whether thresholds have been breached." And the commitments expect change: where approaches evolve, "organisations will provide transparency on this, including their reasons, through public updates" (Department for Science and Technology, 2024). The commitments are voluntary.

62.1.4 What the published frameworks share #

In December 2025 an evaluation organization compared the twelve frameworks then published and counted how many contain each of nine common elements, from capability thresholds to a procedure for updating the policy (Section I.2.2 gives its counts) (Metr, 2025a). The counts record what a document mentions, not how strongly it binds: a commitment phrased as what a developer may do counts as present in such a table (inference).

Model specifications are a different kind of commitment. OpenAI's Model Spec "consists of direct instructions to the model," and Claude's constitution is "written with Claude as its primary audience" (OpenAI, 2025d; Anthropic, 2026r). They commit a model to behavior, where a framework commits a developer to decisions about training and release. Section 19.2 reads them; they enter here only where a framework relies on a model's trained behavior as a safeguard, as when a model "is modified to refuse to provide assistance to harmful tasks" (OpenAI, 2025e).

62.2 The trigger and its buffer #

When does a framework's alarm go off, and can it go off in time? The mechanism the reference frameworks share is simple to state: an evaluation result is compared with a threshold, and crossing it triggers safeguards. Two facts from Chapter 36 make its timing hard. Evaluations are run at intervals, while training continues between them. And a capability evaluation measures from below: it records the best performance the evaluator managed to draw out of the model, not the best anyone could. A framework therefore sets its trigger below the capability it is worried about, and the gap between the two is its buffer. The first framework states its buffer in numbers, which makes the mechanism teachable.

62.2.1 The published design #

The policy states the difficulty itself. "Ensuring that we never train a model that passes an ASL evaluation threshold is a difficult task. Models are trained in discrete sizes, they require effort to evaluate mid-training, and serious, meaningful evaluations may be very time consuming, since they will likely require fine-tuning" (Anthropic, 2023a). Its answer was to evaluate at defined intervals and to design the evaluations "to trigger at slightly lower capability levels than those we are concerned about." Evaluations were to run "after every 4x jump in effective compute, including if this occurs mid-training," and "every 3 months to monitor fine-tuning/tooling/etc improvements" (Anthropic, 2023a). Effective compute is "roughly the amount of compute it would have taken to train a model if no improvements to pretraining or fine-tuning techniques are included," so it rises both with training and with better methods.

The buffer was set against the interval. "We have aimed to set the size of our safety buffer to 6x (larger than our 4x evaluation interval) so model training can continue safely while evaluations take place" (Anthropic, 2023a). The response policy says what happens on a trigger when the next level's protections are not ready: pause training and analyze. An evaluation found "overly conservative (i.e. creating a greater than 6x 'safety buffer')" is rebuilt and the policy updated before training resumes. A model found to have already passed the next level during training means the company should "immediately lock down access to the weights" (Anthropic, 2023a).

The policy names the floor it stands on. "It is not currently possible to truly upper-bound the capabilities of generative models," it says, so evaluations should use "close to our best capabilities elicitation techniques" (Anthropic, 2023a). Section 36.2.1 turns that shortfall into a quantity. Write $g_{\text{el}}$ for the elicitation gap: the factor of training compute that would be worth as much as the methods the evaluator did not try, the best ones anyone will apply. That chapter derives the margin a claim of inability must leave on a score scale, (36.1). The trigger needs the same idea on a compute scale.

62.2.2 When the trigger fires in time #

Figure 62.1 makes the budget visible: every slider moves the detection point right, and the level of concern does not move. Two readings of the published numbers follow (inference). The policy says the buffer exceeds the interval "so model training can continue safely while evaluations take place," which assigns the remaining factor of 1.5 to $f_{\text{run}}$. Read that way, the fourfold schedule leaves nothing for an elicitation gap, and the separate three-month evaluations, which track gains from fine-tuning and tools, carry that burden alone. Read the other way, if training pauses while each evaluation runs, the scheme tolerates a gap below 1.5. In score terms the budget is the margin of (36.1): if the score rises $z_{10}$ points per tenfold increase in compute, a buffer of 6 is about $0.78\,z_{10}$ points, and each factor spends its logarithm.

62.2.3 What the arithmetic assumes #

The bound rests on three assumptions, and the record bears on each. The first is that capability rises steadily with effective compute. If an ability appears abruptly between two evaluations, the measured score can jump from below the trigger to past the level of concern within one interval. The trigger then sits at almost the same compute as the level of concern rather than a buffer below it, and step 2 fails. Whether abilities appear that way is the debate of Section 8.6, and the time-horizon trend of Section 36.3 rests on the same assumption of smooth growth.

The second is that the trigger sits where the design says, one buffer below a level of concern that no one has observed. The developer's own review of its first year reports otherwise. Its evaluations "in one domain were not explicitly designed to establish the 6x scaling buffer mentioned in the previous policy. Since our current techniques aren't capable of giving confident buffer predictions, we instead relied on empirical observations and rough predictions" (Anthropic, 2026h) (developer document). It judged the risk minimal "at present, where capability improvements happen more smoothly with scale." The same review reports that one round of evaluations was completed three days after the three-month interval, and that the interval was extended to six months "to avoid lower-quality, rushed elicitation." It adds that some evaluations "lacked some basic elicitation techniques such as best-of-N or chain-of-thought prompting" (Anthropic, 2026h).

The third is that $g_{\text{el}}$ stays fixed between checks. It does not: methods for drawing capability out of a model improve after it is trained. That is why the first policy added periodic evaluations and anticipated "a more conservative safety buffer to account for pace of development of post-training or elicitation techniques" at higher levels (Anthropic, 2023a). OpenAI's framework states the general point: "we regard any one-time capability elicitation in a frontier model as a lower bound, rather than a ceiling, on capabilities that may emerge in real world use and misuse" (OpenAI, 2025e). A model that recognizes when it is being tested makes $g_{\text{el}}$ larger and harder to bound (Chapter 37). The first policy foresaw the case: an advanced model "could display sufficient awareness to conceal its capabilities under certain testing regimes" (Anthropic, 2023a).

62.2.4 The buffer elsewhere #

The other frameworks keep the buffer and drop the numbers. As of October 2026 Google DeepMind's framework sets alert thresholds meant to flag that a critical level may be reached before the next full assessment, and names the risk that others will elicit more than its own evaluators do (Section I.2.3) (DeepMind, 2026). The European Union's code of practice for the most capable general-purpose models asks for a safety margin against later capability gains and under-elicitation (Section I.1.4) (European Commission, 2025). The margin has moved from one developer's design into a regulatory code without the numbers that would let an outsider check whether it is large enough (inference).

62.2.5 The mechanism under strain #

By 2026 the developer that published the numbers reported that, for biological risks among others, its tests could no longer support a strong argument that risks were low or that they were high, a state it calls a "zone of ambiguity" (Anthropic, 2026ap) (developer document; Section I.2.7 gives the record). In the terms of the derivation, the trigger was meant to fire while the model was still clearly below the level of concern. A zone of ambiguity means the evaluations cannot place the model relative to either line, and the arithmetic has nothing to act on (inference). What a crossing then requires is an argument rather than a reading, which is a safety case (Chapter 52). The precautionary decisions that developers took in this zone for biological risk are Section 59.2.4's.

62.3 Thresholds: capability or risk #

A threshold says when protections must change, but on what is it set, and what justifies its level? The frameworks set theirs on capabilities, and the case that they should be justified by something else is the subject of this section.

62.3.1 Two kinds of threshold #

Leonie Koessler, Jonas Schuett, and Markus Anderljung of the Centre for the Governance of AI drew the distinction in a 2024 preprint (Koessler et al., 2024). A capability threshold describes "AI capabilities beyond which an AI system is deemed to pose too much risk." A risk threshold states "how much risk would be too much," as a limit on the likelihood and severity of harm. Other industries state such limits. The authors report that in US aviation the probability of failure conditions that would prevent continued safe flight and landing should not exceed one in a billion per flight-hour. In the UK nuclear industry a member of the public's risk of death is "unacceptable" above $10^{-4}$ per plant-year and "broadly acceptable" below $10^{-6}$ (Koessler et al., 2024).

Their verdict is not that capability thresholds should go. Risk thresholds "are more principled than capability thresholds, but the main downside is that they are more difficult to evaluate reliably." The authors recommend that companies "(1) define risk thresholds to provide a principled foundation for their decision-making, (2) use these risk thresholds to help set capability thresholds, and then (3) primarily rely on capability thresholds to make their decisions" (Koessler et al., 2024). They list what a stated risk threshold would buy, including that it may "prevent motivated reasoning regarding what level of risk is acceptable." They also list its cost: it "may create an incentive to produce artificially low risk estimates." Regulators, they add, "are the most legitimate actors to define risk thresholds" (Koessler et al., 2024).

The link between the two kinds has a simple consequence (inference). Any capability threshold, combined with the safeguards it requires, accepts whatever risk a model just below it poses. A capability threshold therefore carries a risk threshold whether or not anyone states it, and moving the capability threshold without new evidence about how capability turns into harm moves the accepted risk. A stated risk threshold, with the model that maps capabilities to harm, is what would let an outsider tell those two kinds of revision apart.

62.3.2 What the frameworks choose #

The second key update to the 2025 International AI Safety Report, an international scientific assessment by Yoshua Bengio and colleagues, describes the frameworks as focused "mainly on chemical, biological, radiological, and nuclear risks, cyber capabilities, and autonomous behaviour from AI models" (Bengio et al., 2025c). It finds a "lack of clear methods for defining acceptable risk levels, linking capabilities to those levels, and determining appropriate mitigations." As a result, "risk thresholds are often implicit: they are reflected in their chosen responses, such as restricting access, enhancing monitoring, or delaying deployment, rather than based on measurable evidence of harm" (Bengio et al., 2025c).

The capability side has a strong case, and the frameworks make it. As of October 2026 they state thresholds as capabilities chosen to track the potential for severe harm, and some fold a notion of risk into the wording without attaching a number to it (Section I.2.3) (OpenAI, 2025e; DeepMind, 2026). The European Union's code takes the same form. Its risk tiers are defined in terms of capabilities, and the provider must justify why they keep risk acceptable (Section I.1.4) (European Commission, 2025). In regulatory form, this is close to what Koessler and colleagues recommend: capability thresholds for decisions, with a justification that points to risk (inference).

The risk-first design also exists in writing. Siméon Campos and colleagues at SaferAI, in a 2025 preprint, propose that a developer first "define a risk tolerance" (Campos et al., 2025). Risk modeling then turns it into "Key Risk Indicators" with thresholds, and "Key Control Indicators," measures of mitigation effectiveness that must reach their own thresholds whenever a risk indicator crosses its line (Campos et al., 2025). As of October 2026 we found no published framework of a frontier developer built this way.

Choices of scope are threshold choices too. Between 2025 and 2026 one framework moved persuasion out of its scope, and another added levels for harmful manipulation and for misalignment (Section I.2.3) (OpenAI, 2025e; DeepMind, 2025a; DeepMind, 2026).

So are choices of severity. The first policy and OpenAI's framework both place catastrophe at the scale of thousands of deaths or hundreds of billions of dollars (Anthropic, 2023a; OpenAI, 2025e). California's statute, as enacted in 2025, defines a catastrophic risk as one that could contribute to the death of, or serious injury to, "more than 50 people or more than one billion dollars" in damage from a single incident (Legislature, 2025b) (Section I.1.10). The developer's 2026 risk report says it uses the term "in its plain meaning rather than adopting any specific statutory definition," and addresses statutory definitions "in separate compliance frameworks" (Anthropic, 2026ar) (developer report). The legal definition sits more than an order of magnitude below the frameworks' own, and the framework the public reads need not be the document a statute enforces (inference).

62.3.3 Who sets them #

Where a threshold sits is a specification question with no neutral answer: it decides how much risk to others a developer may impose for the benefits of its models (Chapter 16). The Seoul commitments asked that thresholds be defined with input from home governments (Department for Science and Technology, 2024). As of October 2026 two developers' frameworks say they draw on outside input, including from governments, and the European code asks providers to describe whether and how such input influences whether they proceed (Section I.2.3; Section I.1.4) (OpenAI, 2025e; DeepMind, 2026; European Commission, 2025). We found no framework document that says how outside input set any particular threshold.

62.4 Revisions as evidence #

A framework that its author can revise binds only as far as revising it is costly. Section 61.5 derives the point: a commitment is only as strong as the cheaper of breaking it and rewriting it, and only a party other than the one committed can make either expensive. Every framework read here can be revised by the developer that wrote it, with no outside approval. The record of revisions is therefore the best public evidence on how the commitments behave under pressure, and it supports two readings. On the first, revisions are learning: the early versions were written before anyone had run the evaluations they called for, and experience corrects them. On the second, they are drift, the gradual movement toward what is convenient that Section 53.3 traces in organizations that run hazardous systems. This section sets out the record and asks what would tell the readings apart.

62.4.1 Revision written into the trigger #

Both reference frameworks with a trigger write a revision step into it. The first policy's response to a trigger, when the evaluation is judged "overly conservative," is to construct new evaluations and update the policy before training resumes (Anthropic, 2023a). Google DeepMind's framework says that when a model reaches an alert threshold but remains distant from the critical level, "the response plan may include updating the alert threshold to ensure the safety buffer remains appropriate" (DeepMind, 2026). The same move is right when an evaluation was too conservative and is drift when it was not, and only someone who sees the evidence can tell which (inference).

The duty to explain a revision has grown. The Seoul commitments asked for public reasons (Department for Science and Technology, 2024). As of October 2026 California's statute requires a published justification for a material modification, and the European code asks for a changelog that says how and why a framework was updated, with unredacted access for the AI Office, the body that assesses providers' compliance under the European Union's AI Act (Section I.1.11; Section I.1.4) (Legislature, 2025b; European Commission, 2025). The reasons are becoming public; the evidence behind them reaches, at most, a regulator (inference).

62.4.2 The record #

Section I.2.8 lists the versions of the three reference frameworks through July 2026, with each change as its developer describes it, and reads the four entries that carry most of the evidence. Each of the four fits both readings (inference). A developer's own compliance review disclosed shortfalls it had found itself. It answered the missed evaluation interval and the buffer it could not establish by loosening the rule each broke, and the missing elicitation techniques by tracking such gaps (Anthropic, 2026h). Commitments written as if-then rules were recast as public goals that the developer grades itself, for reasons that included, in the developer's words, a policy environment that "has shifted toward prioritizing AI competitiveness and economic growth" (Anthropic, 2026ap). Thresholds were reworded while the developer reported that its models were approaching them, and whether a rewording raises or lowers the bar is a judgment the published text cannot settle without the evidence on how each wording maps to harm. A deployment rule stated as a score became a rule stated as a judgment, which is harder for an outsider to check and easier to apply to cases the scores did not foresee. Read as learning, each change replaces a rule written before anyone had run the evaluations with one that experience informed; read as drift, each moves a commitment to where the developer can meet it.

62.4.3 Clauses that depend on other developers #

Some commitments are written to depend on what competitors do. As of October 2026 OpenAI's framework lets it adjust its required safeguards if it confirms that another developer released a comparably capable system without comparable safeguards (OpenAI, 2025e). It may do so only if it judges that the adjustment does not meaningfully increase the overall risk of severe harm, acknowledges the adjustment publicly, and keeps its own safeguards more protective, sharing information to validate that claim, a condition it presents as a way "to avoid a race to the bottom on safety" (OpenAI, 2025e). Whether those conditions hold is the developer's own judgment (inference). Google DeepMind's counts the marginal risk its release adds to what is available elsewhere (Section I.2.9) (DeepMind, 2026). The first Anthropic policy took the opposite stance, for the same stated reason. To "avoid a 'race to the bottom'," it excluded other companies' models from its baseline of risk: "just because other language models pose a catastrophic risk does not mean it is acceptable for ours to" (Anthropic, 2023a). Whether the current version keeps that footnote, the policy text read for this chapter does not show.

These are conditional commitments, and their logic is the race model's (Chapter 65): each developer's restraint depends on the others' (inference). A clause keyed to marginal risk is coherent for a single release, since a model no more dangerous than what is already available adds little harm. Applied by every developer to every release, it lets the weakest safeguards in the market set the baseline for all (inference). As of October 2026 we found no public statement by a developer invoking such a clause.

62.4.4 What a revision can show #

A revision shows the text before and after, its date, and the reason given. It cannot show whether the reason was the cause, what evidence lay behind it, or whether a decision would have gone differently under the old text. Two observations would separate the readings (inference). Drift predicts that thresholds are rewritten as models approach them, in the direction that leaves the models below; learning predicts revisions in both directions, unrelated to where the models stand. And learning predicts that the stated reasons cite evidence an outside reviewer has seen. The record as of October 2026 fits both (Section I.2.8). The 2026 threshold rewrites came, by the developer's own account, as "model capabilities have clearly approached the RSP thresholds" (Anthropic, 2026ap); the same period added a threshold for state programs, a power of external review, and protections activated before a threshold was shown to be crossed. On present evidence the record does not decide between learning and drift.

62.5 Frameworks as self-regulation #

From regulatory theory. How should a state govern firms that know more about their risks than it does? Regulatory theory's answer for one class of problems is to make the firm plan, and to regulate the planning. The frameworks are a case of it, first voluntary and now, for the largest developers in some jurisdictions, required by law.

62.5.1 Management-based regulation #

Cary Coglianese and David Lazer named the strategy in a 2003 article in the Law & Society Review (Coglianese and Lazer, 2003). Regulation can act at three stages of a firm's activity. Technology-based rules act on what the firm does, performance-based rules on what it produces, and management-based regulation on how it plans. In their words, it "directs regulated organizations to engage in a planning process that aims toward the achievement of public goals, offering firms flexibility in how they achieve public goals." Firms produce plans meeting general criteria, such as "the identification of hazards, risk mitigation actions, procedures for monitoring and correcting problems." The authors note that the strategy resembles what others call "enforced self-regulation," citing John Braithwaite's 1982 article (Coglianese and Lazer, 2003; Braithwaite, 1982).

When does the strategy fit? Performance standards need a regulator who can measure outputs, and technology standards need firms similar enough that one prescription suits them all. Where the regulator "is neither capable of measuring output nor can it develop an appropriate technology standard due to industry heterogeneity," the authors write, "a strong theoretical justification exists for management-based regulation" (Coglianese and Lazer, 2003). Frontier AI meets both conditions (inference): the outputs of concern are rare catastrophes whose absence proves little, and the developers and their systems differ and change every generation.

The authors model the firm's choice in two stages, and the model says what a state must do. At the first stage the firm decides whether to plan, at a cost; at the second, having learned what its plan would cost and gain, it decides whether to carry the plan out. Whether either step needs a mandate depends on the firm's own incentives at each stage, which gives four cases (Table 62.1).

Case C carries a warning. If the state mandates only implementation, a firm that discovers a costly but beneficial measure "will have the incentive to hide this information from the regulator" (Coglianese and Lazer, 2003). In cases C and D the regulator therefore needs three capacities: "(1) the planning process to be transparent enough so that the regulator can independently evaluate the data; (2) the technical capacity to evaluate the data and determine whether the plan the firm proposes is consistent with the data; and (3) the capacity to evaluate implementation of the plan." Case D, which needs both mandates, is "likely the most common scenario" (Coglianese and Lazer, 2003).

62.5.2 The record of a precedent #

The authors' main case is food safety. Since the late 1990s, United States regulators have required meat, poultry, and seafood processors to analyze the hazards at each stage of production, set limits at critical control points, and monitor and correct them, a scheme called HACCP (Coglianese and Lazer, 2003). The early record, from government audits the paper reports, is mixed. The seafood regulator could inspect processors only once a year, and in about half of inspections the product line chosen was not running, so inspectors reviewed paperwork. Three years in, "a majority of fish processors still have plans that are not in compliance." In in-depth reviews of 47 meat and poultry plants in 2000 and 2001, 44 had significant violations, 42 of them because of an incomplete hazard analysis (Coglianese and Lazer, 2003). The meat regulator, despite inspectors present at every plant, could evaluate "implementation of a HACCP plan, and not the design of the plan itself." The outcome data were more encouraging. Salmonella prevalence fell by 10 to 28 percent across product lines after the rules took effect, and the combined incidence of seven tracked bacterial illnesses fell 23 percent between 1996 and 2001, which the authors call suggestive (Coglianese and Lazer, 2003).

The authors name the deeper problem. Firms may see little point in "including in the plan any hazards that government inspectors are unlikely to spot on their own." So "the very instances in which a firm's expertise would help it to identify hidden hazards may well be some of the same ones in which the firm has the opportunity and incentive to keep its hazards hidden" (Coglianese and Lazer, 2003). The authors conclude that the strategy needs "a governmental enforcement presence." Enforcement, they add, "may be made more difficult because the same conditions that make it difficult for government to impose technological and performance standards may also tend to make it more difficult for government to determine what constitutes 'good management'" (Coglianese and Lazer, 2003).

62.5.3 What the statutes turned the frameworks into #

The voluntary frameworks assume case A: that developers will plan and carry out their plans without a mandate (inference). The Seoul commitments were voluntary, and much of what the frameworks specify still is (Department for Science and Technology, 2024; Bengio et al., 2025c). One developer lists California's statute, New York's, and the European code among the government measures that now require frameworks (Anthropic, 2026ap). California's law, the one read here, moves the developers it covers to case D (inference). The law, which Section 61.5 reads as a commitment device, requires a large frontier developer to "write, implement, comply with, and clearly and conspicuously publish" a framework (Legislature, 2025b), and Section I.1.11 sets out what the framework must contain as of October 2026. The statute says what a framework must address, not where its thresholds must sit, and it requires no approval of the plan. The European code asks for a framework with justified risk acceptance criteria (Section I.1.4) (European Commission, 2025). In the terms of the field, this is management-based regulation with a planning and an implementation mandate, the family Coglianese and Lazer place enforced self-regulation in (inference). Section 63.2 analyzes the statutes. The escalation from persuasion to sanction that Ian Ayres and John Braithwaite's Responsive Regulation is known for is Section 63.3's subject. Its abstract, the part of the book read here, argues that regulation "does not have to proceed with an adversarial tone, nor does it have to be 'soft' or 'hard' to be effective" (Ayres and Braithwaite, 1992).

62.5.4 Soft and hard commitments #

Why would developers choose commitments this soft, and why would states accept them? Kenneth Abbott and Duncan Snidal's 2000 article, read here in its abstract, gives the dimensions (Abbott and Snidal, 2000). Hard legalization means "precise, legally binding obligations with appropriate third-party delegation"; softer forms combine "reduced precision, less stringent obligation, and weaker delegation." Soft legalization, they argue, "is easier to achieve, provides strategies for dealing with uncertainty, infringes less on sovereignty, and facilitates compromise among differentiated actors" (Abbott and Snidal, 2000).

Rated on the three dimensions, the frameworks as of October 2026 are soft (inference). Obligation is low: they are voluntary and revisable by their authors, one developer's roadmap goals are explicitly not commitments, and one framework lists what its developer "may utilize" (Anthropic, 2026ap; xAI, 2025c). Precision varies. OpenAI's Critical cyber threshold names concrete achievements, while Google DeepMind's first machine learning research level is a model whose use results in "AI progress substantially accelerating from historical rates" (OpenAI, 2025e; DeepMind, 2026). Delegation is nearly absent. No third party decides whether a threshold was crossed in any developer's framework, and the one framework that requires an outside review before resuming is written as guidance (Shanghai AI Laboratory and AI, 2025). The statutes raise obligation for the firms they cover and delegate enforcement to an attorney general, but they add little precision and no delegation of the judgment that matters (inference). Uncertainty, one of the abstract's reasons for choosing soft law, is among the reasons the developers give for revising.

62.5.5 Collective self-regulation #

If any developer's accident harms all of them, the industry has a reason to police itself together. Jide Alaga and Jonas Schuett's 2023 preprint proposes coordinated pausing: when a model fails a set of evaluations, its developer pauses, others are notified and pause related work, and all resume once safety thresholds are met (Alaga and Schuett, 2023). They discuss four versions, from purely voluntary pausing under public pressure, through a collective agreement and a single shared auditor, to a legal requirement, and they find "a number of practical and legal obstacles," "especially how to avoid violations of antitrust law" (Alaga and Schuett, 2023). The Anh Han and colleagues' paper "Voluntary safety commitments provide an escape from over-regulation in AI development" (Han et al., 2022), and Joseph Rees's Hostages of Each Other: The Transformation of Nuclear Safety since Three Mile Island (Rees, 1994), could be read here only as catalog records. Christine Parker's The Open Corporation, read in its abstract, argues that "law and regulators needed to focus much more on 'meta-regulating' corporate self-regulation" (Parker, 2002). Cary Coglianese and Evan Mendelson's review, also read in its abstract, places meta-regulation and self-regulation as alternatives to command-and-control rules, each with strengths and weaknesses that depend on policy conditions (Coglianese and Mendelson, 2010). The recombinant DNA scientists' scale of containment matched to risk is Section 64.2's (Berg et al., 1975).

62.5.6 Where the analogy stops #

Management-based regulation assumes that the regulator can judge whether a firm's plan is adequate, and the food-safety record shows that even a regulator with inspectors in every plant could judge implementation and not design. For a frontier developer the design is the threshold and the evaluations behind it, and the hidden-hazard problem is sharper (inference). The firm builds most of the instruments that would reveal the hazard and, short of the EU's compelled evaluations, decides when outsiders test its models: OpenAI's framework, for one, brings in third parties "if we deem that a deployment warrants deeper testing" (OpenAI, 2025e). Every such instrument measures from below (Section 62.2). The product changes faster than a review cycle: one developer's framework went through nine versions between September 2023 and July 2026. And the capability being regulated may respond to its measurement (Chapter 37). As of October 2026 we found no study that tests whether a regulator can judge a frontier developer's threshold choices when the evaluations behind them are the developer's own.

62.6 Frameworks as corporate governance #

From corporate governance. A framework allocates decisions inside the firm: who determines that a threshold was crossed, who may overrule that determination, and who hears of a violation. The theory of boards and the corporate forms developers use are Section 53.6's subject; this section is about the decision rights the frameworks create.

62.6.1 Who decides, in the documents #

As of October 2026 each reference framework places these decisions inside the developer, and Section I.2.10 sets out each allocation as written. In OpenAI's, a Safety Advisory Group of company leaders recommends, leadership decides, and the board may reverse a decision (OpenAI, 2025e). Anthropic's first policy named a Responsible Scaling Officer who reports each quarter to the board and to the company's trust (Anthropic, 2023a), and a later version let the trust request external review of risk reports (Anthropic, 2026h). Google DeepMind's assigns decisions to "the appropriate governance function" (DeepMind, 2026). In all three documents, the people who determine whether a threshold was crossed are employees, and the final decision rests with company leadership or an internal body (inference).

62.6.2 Three lines of defense #

Several of the governance designs proposed for developers come from Jonas Schuett and colleagues at the Centre for the Governance of AI. Schuett's 2025 article in AI & Society adapts the three lines of defense, a risk-management model "considered best practice in many industries, such as finance and aviation," to AI companies (Schuett, 2025b). The first line "provides products and services to clients and manages the associated risks." The second "provides complementary expertise and support, but also monitors and challenges risk management practices." The third, internal audit, "provides independent and objective assurance" to the board (Schuett, 2025b). A companion article in Risk Analysis argues that frontier developers need such an internal audit function, "organizationally independent from senior management" and reporting directly to the board (Schuett, 2025a). A third, with Ann-Katrin Reuel and Alexis Carlier, sets out the design choices for an ethics board, among them how it makes decisions and whether they bind (Schuett et al., 2025). In a 2023 survey of 51 experts from developers, academia, and civil society, Schuett and colleagues found that respondents somewhat agreed that developers should have a board risk committee, a chief risk officer, and an internal audit team (Schuett et al., 2023) (preprint). "Based on public information," they add, "AGI labs do not seem to have any of these structures." The authors note that their sampling frame "was likely biased towards experts that were known to the author team." An earlier design for internal auditing across the development of a system came from authors at Google and the Partnership on AI (Raji et al., 2020).

Mapped onto the frameworks, OpenAI's advisory group is a second-line function, appointed by and advising the leadership it checks; Anthropic's scaling officer is a member of staff who reports to the board; and none of the documents names an internal audit function (inference). The model's own record warns against reading much into the mapping. Schuett reports the standard criticisms. The first line's incentives favor profit over risk, the second line often lacks independence and expertise, and the model can give "a false sense of security." And "the second line might not recognize that they only see what the first line chooses to show them" (Schuett, 2025b). And he finds that "there do not seem to be any (high-quality) studies on the effectiveness of the 3LoD model" (Schuett, 2025b).

62.6.3 The predictions to test #

Two predictions from organization theory frame the evidence. Pledges to stakeholders that their makers have no incentive to keep mostly do not bind, and formal structures adopted for legitimacy can be decoupled from practice (Bebchuk and Tallarita, 2020; Meyer and Rowan, 1977). Section 53.6 and Section 53.7.1 report both arguments, and the second names the observable: whether a developer invites or avoids outside inspection of the occasions on which its framework met a decision. The last channel is the employee who objects. The letter in which current and former employees asked for a right to warn, and the statute's protection for those who report, are Section 53.4.1's (Current and companies, 2024).

Where the analogy stops: governance theory assumes the board can evaluate what management reports. On whether a threshold was crossed, the people best placed to judge report to the management whose plans the judgment constrains, and a board told everything may still be unable to assess an evaluation's elicitation gap (inference).

62.7 What outside reviewers found #

What have people outside the developers found when they read the frameworks? Each outside reading is itself a document by a party with a position, and the frameworks are disclosures by an interested party, which Section 5.6 teaches the reader to mark. Section I.5.4 lists the reviews with their provenance, method, and main findings, as of December 2025.

62.7.1 What the reviews say, and who says it #

The international assessment states its conclusion with care: the effectiveness of the measures "remains uncertain, and significant disagreement exists about whether current approaches are appropriate or sufficient" (Bengio et al., 2025c). On voluntary measures it notes that "this may increase adaptability, but may also be a limitation," and it cites one study for the finding that companies have fulfilled earlier voluntary commitments inconsistently (Bengio et al., 2025c).

That study, by Jennifer Wang, Kayla Huang, Kevin Klyman, and Rishi Bommasani, scored the companies that signed the White House's 2023 commitments on what they had disclosed (Wang et al., 2025c) (preprint). The six highest scorers were the six members of the Frontier Model Forum, an industry body. The authors identify a design flaw rather than a moral one: "companies made public commitments to the White House, but no mechanism was created to monitor implementation or provide the public with information about implementation" (Wang et al., 2025c). The method carries its own counter-reading. A score built from public disclosures cannot tell a measure not taken from a measure not disclosed, which the authors acknowledge when they describe such work as limited "due to the lack of transparency into company-internal practices" (Wang et al., 2025c). The commitments scored were the earlier ones, not the frameworks, but the finding bears on what voluntary commitments without monitoring produce.

The Future of Life Institute's index is a panel's judgment, not an audit (Future of Life Institute, 2025). Evidence was gathered from March 24 to June 24, 2025, through desk research and a company survey; panel members, chosen for expertise and "absence of conflicts of interest," graded each domain, and their individual grades remain confidential. The organization states its aim as "to strengthen incentives for responsible AI development and to close the gap between safety commitments and real-world actions." Section I.5.4 gives its 2025 grades and findings.

METR's comparison is a compilation, not an assessment of compliance, by an organization that has also reviewed one developer's risk reports (Metr, 2025a; Bowman et al., 2025). The developer-commissioned reviews are the closest thing to an audit in the record. Section 52.4 reports the first: an internal team and METR reviewed a pilot risk report, saw evidence beyond what was published, and endorsed its assessment with reservations. A pilot in which two developers evaluated each other's models is Section 36.8's (Anthropic, 2025x). In each, the developer chose or approved the reviewer and the scope (inference), which Section 38.6 treats as the central design question for paid verifiers.

62.7.2 Decisions the developers attribute to thresholds #

As of October 2026 the record holds several decisions that developers say their frameworks produced: safeguards added to models released in 2025 after pre-deployment testing could not rule out meaningful help to novices developing biological weapons (Bengio et al., 2026), a pause in reinforcement learning training in 2026 that one developer attributed in part to a model that may meet a Critical cybersecurity threshold (OpenAI, 2026q), the same developer's determination in September 2026 that the model had reached that threshold, which it reports led to its strongest safeguards yet and a leadership determination that they sufficed for public launch (OpenAI, 2026l), and a published determination in February 2026 that a model did not cross an AI research threshold (Anthropic, 2026h). Section I.2.11 gives the record.

Read for what they show, these are the frameworks meeting decisions, in the developers' accounts (inference). Read for what they cannot show, each decision rests on a judgment made under uncertainty by the party that holds the evidence, and the developer is the only witness to what it would have done without the framework. In the pause the developer names the threshold as one reason among three, so even its own account does not isolate the framework's effect (OpenAI, 2026q).

62.7.3 The counter-reading every framework invites #

A framework whose thresholds are never crossed, or whose crossings are not published, fits two worlds: safe models, and tests too weak to find danger. Safety benchmarks that track general capability let capability gains be reported as safety progress, the finding Section 5.2.2 reports (Ren et al., 2024b), and an absence claim is worth only the search behind it (Section 5.6.2). The observable that separates the readings, by the decoupling argument, is whether outsiders can inspect the link between a framework and a decision (inference). As of October 2026 the record runs both ways: developers invited pilot reviews and published risk reports, and in the same period the power to request external review went unused and the required internal circulation of unredacted reports was narrowed to at least 200 employees (Anthropic, 2026ar; Anthropic, 2026h).

62.8 What the record means for delegation #

The book's thesis is that alignment is a delegation problem under an asymmetry of competence and visibility, and one level up the developer is the public's agent. In the ledger of agency costs that Section 61.1 draws, a published framework is bonding: what the agent spends to make its own promises credible. The older structures fit closely (inference). A framework is a management plan, the statutes are management-based regulation with planning and implementation mandates, the proposed governance designs are the three lines of defense, and coordinated pausing is collective self-regulation. Their predictions are consistent with what can be checked: the public evidence that companies kept voluntary commitments without monitoring varied widely (Wang et al., 2025c), though what they did without disclosing it is unobserved, and the documents give the first and second lines the decisions and no one the third.

What breaks is specific (inference). The regulated party builds most of the instruments that measure the hazard and controls most outsiders' access to its models, and every instrument gives a lower bound whose gap grows after the test (Section 62.2). The product changes faster than any review (Section 62.5.6). The measured capability may respond to the measurement, which is evaluation awareness, one of the results the book listed in advance as having no counterpart in the older fields (Section 1.7.1). And the agent also writes the scale of harm the commitment protects against, setting catastrophe more than an order of magnitude above the level a statute uses (Section 62.3.2).

Almost every claim in this chapter concerns the third relationship, the developer and the public. The safeguards a framework triggers act on the second, the deployed system and the people it serves. Almost all of the evidence is the developers' own documents and reports, with a few outside reviews of what those documents disclose. The question the chapter set, whether a self-written and self-revisable commitment constrains its author when keeping it is costly, therefore remains open. The documents show what was promised and how it changed, and no one outside the developers has yet seen enough to say whether the promises bound.

62.9 Settled, contested, missing #

62.10 Exercises #

Further reading #

• Anthropic (2023a): the first framework, with its trigger and buffer stated in numbers.
• Anthropic (2026h): one framework's version history, the record this chapter reads as evidence.
• Department for Science and Technology (2024): the commitments that made publishing a framework the norm.
• Coglianese and Lazer (2003): regulating a firm's management of risk rather than its outcomes, with the conditions under which it works.
• Abbott and Snidal (2000): why actors choose soft commitments under uncertainty.
• Braithwaite (1982): enforced self-regulation, by its title the founding statement of the strategy the statutes now follow.