Draft complete but not yet reviewed; expect revisions

The Laboratory as the Public's Agent

Most of this book asks how a developer can make a model do what the developer intends, and how the developer can tell whether it does. The previous part ended with what deployed systems do to the people who use them (Chapter 60). This part asks the same questions one level up. The public, through its governments and markets, has let a few laboratories build a technology whose risks fall partly on people who are not their customers. The laboratories know far more about their systems than any outsider, they act largely out of sight, and outsiders often cannot judge the quality of their safety work even after it is done. Current and former employees of frontier developers wrote in 2024 that the companies "possess substantial non-public information about the capabilities and limitations of their systems" (Current and companies, 2024). That is the book's thesis condition, delegation under an asymmetry of competence and visibility (Section 1.7.1), and it holds here without any claim about what a model wants or believes (Section 1.3). The delegation models of Chapter 14 fit this relationship more directly than the other two, because a laboratory, unlike a model, has aims of its own before anyone contracts with it and can refuse the contract (Section 14.8.2).

The chapter asks what kind of delegation this is and which institutions could make the agent act for its principal. It first states what the delegation costs, in the terms economics uses for any agency relationship, and why the public's position is the hardest case of it. It then sets out the claim that organizes this part, that the book's four problems recur when the developer is the agent, and asks who the developer actually answers to, since investors, staff, customers, and governments all pull on it at once. Two institutions that claim to close the gap come next: professional norms, which govern doctors and auditors, and corporate forms and charters, which developers have used to bind themselves. The chapter ends with the objections the relationship invites: capture, the legitimacy of a private body writing rules for everyone, and the risk that a well-aligned system makes its controller more powerful.

The formal results here are theorems within their models. The record on the laboratories themselves is mostly their own descriptions, a few statutes, and outside ratings, so most of what the chapter says about developers is the book's inference and is marked as such. Every claim concerns the third of the book's relationships, the developer and the public. Safety frameworks, regulation, earlier technologies, competition, international coordination, transparency, and power each get a chapter of their own.

61.1 The public delegates to the laboratory #

What does the public lose by acting through laboratories, and what can it spend to lose less? Economics calls the party that hands over a task the principal and the party that carries it out the agent (Section 14.1). Here the developer is the agent and the public a principal that cannot write the contract, cannot observe the work, and often cannot judge the result. This section teaches two tools that measure what such a delegation costs: an accounting of agency costs, and the economics of services whose quality the buyer cannot check. It then asks how much discretion a principal in this position should grant.

61.1.1 What delegation costs #

Michael Jensen and William Meckling's 1976 theory of the firm, in the Journal of Financial Economics, defines an agency relationship as "a contract under which one or more persons (the principal(s)) engage another person (the agent) to perform some service on their behalf which involves delegating some decision making authority to the agent" (Jensen and Meckling, 1976). If both parties pursue their own interests, the authors reason, the agent will not always act in the principal's. They define agency costs, what the principal loses by acting through such an agent, as the sum of three parts: "the monitoring expenditures by the principal," "the bonding expenditures by the agent," and "the residual loss" (Jensen and Meckling, 1976).

Each part has a precise meaning. Monitoring, in a footnote, includes more than watching: it covers efforts to control the agent through "budget restrictions, compensation policies, operating rules" (Jensen and Meckling, 1976). Bonding is what the agent spends to guarantee that it will not take actions that harm the principal, or that the principal will be compensated if it does. The authors' examples are guarantees to have the accounts audited by a public accountant, explicit bonding against malfeasance, and "contractual limitations on the manager's decision making power" (Jensen and Meckling, 1976). The residual loss is the dollar value of the divergence between the agent's decisions and those that would serve the principal best, which remains after monitoring and bonding have done what they can.

The theory's central result concerns who pays. When an owner-manager sells shares to outsiders, "the owner will bear the entire wealth effects of these expected costs so long as the equity market anticipates these effects" (Jensen and Meckling, 1976). Buyers foresee that the manager's interests will diverge from theirs and pay less for the shares. Because the agent bears the agency costs, the agent has its own reason to bond: it pays the owner-manager to bond and to accept monitoring as long as the marginal benefit of each exceeds its marginal cost (Jensen and Meckling, 1976).

Table 61.1 maps the three parts onto the public and a developer. The mapping is the book's, and each later chapter of this part is about one of its lines (inference).

The residual-loss row is where the public's case departs from the firm's (inference). In the firm, a market prices the agent's expected behavior and so moves the agency costs onto the agent. No market prices a developer's expected behavior toward the public. Customers price what they buy and investors what they own, while a person harmed who is neither has nothing to discount. Anthropic's own account of its governance states the point. Customers, workers, and suppliers can negotiate terms that reflect the costs and benefits of their exchanges. "But other parties, such as the general public, don't directly contract with a corporation and therefore do not have a means to charge or pay for the costs and benefits they experience" (Anthropic, 2023f) (developer document). In Jensen and Meckling's terms, the residual loss to the public stays with the public unless law or reputation moves it back to the developer. The agent's own reason to bond, which in the firm comes from bearing the cost, weakens accordingly (inference). Whatever leads developers to publish commitments must then come from elsewhere: from the prices that customers and investors do set, from staff, from anticipated regulation, or from motives the model leaves out. The later sections take up each.

61.1.2 The hardest case: a credence good #

From economics. The public's difficulty is not only that it cannot watch the developer work. It often cannot tell, even afterward, whether the work was done well. Economists call a service of that kind a credence good: one whose quality the buyer cannot judge even after receiving it, such as a car repair or a medical treatment. It is the limiting case of hidden information, in which the agent knows something about the task that the principal does not (Section 14.1.3). Michael Darby and Edi Karni's 1973 title names the question it raises, the optimal amount of fraud under free competition (Darby and Karni, 1973). Kenneth Arrow's 1963 paper on uncertainty and the welfare economics of medical care is older still (Arrow, 1963). Both could be read only as catalog records, so their arguments are not reported here.

The readable theory is Uwe Dulleck and Rudolf Kerschbamer's 2006 survey in the Journal of Economic Literature, which unifies the field in one model. Its question: "Under which conditions do experts have an incentive to exploit the informational problems associated with markets for diagnosis and treatment?" (Dulleck and Kerschbamer, 2006). In the model, each customer has either a minor problem, which a cheap treatment fixes, or a major one, which only an expensive treatment fixes; the expensive treatment fixes either. The customer knows the probability of the major problem but not which one he has, and the expert learns it by diagnosis. Three kinds of fraud are possible: undertreatment, the cheap treatment for a major problem; overtreatment, the expensive treatment for a minor one; and overcharging, charging for the expensive treatment while providing the cheap one (Dulleck and Kerschbamer, 2006). The customer is satisfied in three of the four combinations of need and treatment. "Only in one case, where he has the major problem but gets the cheap treatment, will he discover ex post what he needed and what he got" (Dulleck and Kerschbamer, 2006).

Two institutions can close the gap. Under liability, an expert "cannot provide the cheap treatment" if the expensive one is needed; under verifiability, an expert "cannot charge for the expensive treatment" if she provided the cheap one (Dulleck and Kerschbamer, 2006). The survey adds two assumptions: that customers are alike (homogeneity), and that a customer who has been diagnosed is committed to treatment by the same expert (commitment). Its first proposition states that under homogeneity, commitment, "and either L (Liability) or V (Verifiability) or both, market institutions solve the fraudulent expert problem at no cost" (Dulleck and Kerschbamer, 2006). The derivation below shows why, in the model's simplest form.

The result carries its assumptions with it, and the authors list them. They note that "strictly speaking Assumption H is never satisfied in practice," that strict liability is difficult to impose for repairs and for medicine, and that verifiability is "a too demanding assumption with medical treatments" (Dulleck and Kerschbamer, 2006). They also note that the literature had ignored differences among customers in their expertise, although some customers can verify what was done and others cannot (Dulleck and Kerschbamer, 2006). The public facing a developer is a customer of the second kind (inference).

That difference was measured in the field by Loukas Balafoutas, Adrian Beck, Rudolf Kerschbamer, and Matthias Sutter, a group that includes one of the theory's two authors. The study was published in 2013, and its working-paper version is the one read here (Balafoutas et al., 2013). They chose taxi rides because confounds present in other credence markets, such as misdiagnosis, incompetence, or overly cautious experts, are practically absent there. Three experimenters took 174 rides in Athens, departing one by one from the same place to the same destination, with a GPS logger recording each route. In each triple, one passenger, speaking Greek, only stated the destination, while the other two said they were not familiar with the city, one of them speaking English (Balafoutas et al., 2013). The average ride was 12.9 kilometers, of which 1 kilometer, 8 percent, was an unnecessary detour. Passengers who revealed that they did not know the city were taken on detours of double length, and in 11 percent of rides passengers were overcharged through incorrect tariffs: 19 percent for English speakers against 7 percent for Greek speakers (Balafoutas et al., 2013).

The theory predicted the pattern. A taximeter makes the route verifiable and paying on arrival works as liability, so in Dulleck and Kerschbamer's discussion the only problem left is a detour. A driver has reason to take one when "her passenger is a stranger in the city" (Dulleck and Kerschbamer, 2006). The tariff was the part a stranger could not verify, and there the drivers overcharged. The result is one experiment in one city's taxi market, by a group that includes an author of the theory it tests, so it is reported rather than settled. Its strongest limit is scale: it shows that individual sellers exploit a visible lack of information when the stakes are a fare, not how a large firm behaves under regulation and reputation.

What the model implies for a developer is the book's reading (inference). Treatment is the safety work done before and after a release, and a major problem is a model whose risks need more than routine mitigation. The price the developer charges the public is the credit it receives, in trust and in room to regulate itself, for the safety work it says it did. Liability is weak for the harms that matter most, since catastrophic harm can exceed a firm's assets and be hard to trace to its cause (Section 63.1; Section 63.4). Verifiability is partial: capability evaluations give a lower bound on what a model can do (Section 36.2), and disclosure theory predicts that an informed party reports selectively when outsiders cannot tell what it knows (Section 5.6.1). With both institutions weak, the model predicts both failures of its fourth proposition: undertreatment, less safety work than the risk requires, and overcharging, credit claimed for safety work not done. One route to the second, which a 2024 preprint identifies, is a safety benchmark that tracks general capability, potentially letting capability gains be reported as safety progress (Ren et al., 2024b) (preprint; Section 5.2). The model also names the remedy: either liability that reaches the developer, or records of what safety work was done that someone other than the developer can check (Chapter 38).

The analogy stops in three places (inference). In the model the customer pays and bears the outcome; the public bears harms from trades between the developer and its customers, so a market that served customers honestly would still say nothing about third parties. The model assumes that the expert knows which problem the customer has, and the taxi study chose its market to exclude misdiagnosis. A developer often does not know whether its model's risk is major, which is the subject of Chapter 36 and Chapter 37, so the frontier case adds honest error to the incentive to cheat. And the public sees no posted prices from which to read the developer's incentives.

61.1.3 How much discretion to grant #

From political science. Legislatures have long delegated to agencies that know more than they do, and political science has studied how much discretion such a principal should grant. David Epstein and Sharyn O'Halloran's Delegating Powers, read here in its abstract, examines major US policy initiatives from 1947 to 1992 and describes "the conditions under which the legislature narrowly constrains executive discretion, and when it delegates authority to the bureaucracy" (Epstein and O'Halloran, 1999). The abstract does not state the conditions, so they are not reported here.

Kathleen Bawn's 1995 paper in the American Political Science Review, also read in its abstract, states the trade-off at the center of the question. In practice, "agency procedures designed to promote technical competence often allow for political drift, and those that promote political control provide little new technical information about the consequences of policy decisions" (Bawn, 1995). In her model a legislative coalition chooses the agency's expected preferences and its independence, and "depending on exogenous levels of technical and political uncertainty, optimal agency procedures can maximize technical competence, maximize political control, or achieve a combination of the two" (Bawn, 1995). A review by Sean Gailmard and John Patty, by its abstract, describes how formal models of bureaucracy moved from treating it as "a device for making policy commitments last" to the information asymmetries between agencies and the politicians they serve (Gailmard and Patty, 2012). Two works central to the field could be read only as catalog records: John Huber and Charles Shipan's Deliberate Discretion? and Terry Moe's 1984 paper on the new economics of organization (Huber and Shipan, 2002; Moe, 1984).

Applied to developers, the trade-off reads as follows (inference). Self-regulation is wide discretion, chosen for technical competence at the price of drift, at a time when outsiders were most uncertain about the technology. As uncertainty about developers' aims rises relative to uncertainty about the technology, Bawn's model moves the best procedure toward political control, even at the cost of technical information. California's 2025 statute requiring large developers to publish and follow their frameworks, and whatever else Section 63.2 reports, are consistent with that shift, but consistency with a model is not a test of it. As of October 2026 we found no study that measures distrust of developers and relates it to the AI statutes. The model also names the price: procedures built for control "provide little new technical information."

Where the analogy stops: Bawn's legislature chooses its agency's expected preferences and independence through the procedures it designs. The public does not choose a developer's preferences, and a developer is a private firm with owners of its own, not a bureau that a legislature created (inference).

61.1.4 Which relationship, and where the ledger stops #

Everything in this chapter concerns the developer and the public. One asymmetry that reverses for models does not reverse here. A developer can copy its model, rerun it on the same input, and inspect its weights, which gives it more visibility of behavior than any employer has. The public can do none of these things to a developer. Section 14.8.2 records that the hidden-action model, in which the principal sees outcomes but not the agent's actions, holds for this relationship because no one reruns a laboratory.

The ledger stops where the public's position differs from an investor's (inference). Jensen and Meckling's principal chose to delegate: the outside shareholder bought the shares and can sell them. The public signed no contract with any developer, and its means of ending the relationship are law and the market, both slow.

61.2 The four problems one level up #

The claim that organizes this part is that the four problems the book poses for a model recur, problem by problem, when the developer is the agent and the public the principal, and that the delegation literatures speak to the developer as directly as they speak to the model. The claim is a framing, the book's own, and this section states it so that the later chapters can be read against it (inference).

Specification, saying what is wanted, is done by the agent. The developer writes its own mission, charter, safety framework, and model specification (Section 19.2), and the public cannot write them. The closest precedent is the regulatory strategy that Cary Coglianese and David Lazer named management-based regulation, which directs a firm to plan how it will meet public goals, quoted in Section 1.3 (Coglianese and Lazer, 2003). Whether a specification written by a developer can be legitimate is Section 17.5's question.

Instillation, getting the agent to acquire what was specified, is a matter of the incentives and culture that make a developer's staff act on its written commitments, which Chapter 53 examines. Verification is outside evaluation and audit, disclosure, and whistleblowing. Inioluwa Deborah Raji and colleagues conclude that audits alone are unlikely to achieve accountability and that sustained focus on institutional design will be required, as Section 1.3 quotes (Raji et al., 2022). Staff have asked for protection when they warn of risks (Section 53.4.1) (Current and companies, 2024). And a field experiment in which a package that included random assignment, central pay, and backchecks made pollution auditors report more truthfully is recalled from Section 38.6.1 (Duflo et al., 2013). Maintenance under pressure, keeping commitments as the pressure on them grows, is a matter of competition, whose effect on investment in safety the race models of Section 65.2 study (Armstrong et al., 2016), and of capture, the subject of Section 61.6.

The analogy stops at manufacture (inference). A model is made by its principal; a developer is not made by the public. The public's leverage over a developer's dispositions is far weaker than a developer's over its model's, so instillation one level up is mostly selection and sanction, the two strategies Section 14.5.2 contrasts, rather than training. And even selection is weak here, since the public did not choose which organizations would build the technology.

61.3 Many principals, one developer #

"The public" is a simplification, as "the principal" was for a model (Section 1.4). This section replaces it with the parties that actually act on a developer, the instrument each uses, and the people left out. The theory is common agency, one agent serving several principals at once, the subject of a 1986 paper by Douglas Bernheim and Michael Whinston (Bernheim and Whinston, 1986a) and taught in Section 14.4. With complete information, Avinash Dixit, Gene Grossman, and Elhanan Helpman prove that the principals' equilibrium in truthful strategies implements an action efficient for the principals and the agent together (Dixit et al., 1997). Anyone with no offer to make has no instrument in it and counts only as far as the agent or a principal values that person's welfare, the book's reading in Section 14.4.1 (inference). In the authors' own application the government values the welfare of all individuals, "lobbying or not" (Dixit et al., 1997). The weakening of incentives by several principals, in the book's two-principal model, comes entirely from the agent's dislike of risk (Section 14.4.2), so it is not carried over here without that condition.

Four parties act on a frontier developer. Investors act through capital and board seats; staff through their labor, their equity, the threat of leaving, and their voice; customers through revenue and the terms they accept; and governments through law, procurement, export control, and requests for access. One developer's description of default corporate governance shows how the first instrument is built. Directors are elected and removed by stockholders and legally accountable to them, and "the rights to elect, remove, and sue directors belong exclusively to the stockholders." In the developer's words, "all the key mechanisms of accountability in corporate law push directors to prioritize the financial interests of stockholders" (Anthropic, 2023f) (developer document). The vocabulary for the second comes from the title of Albert Hirschman's book, which names exit and voice as responses to decline in firms, organizations, and states (Hirschman, 1970). Figure 61.1 draws the structure.

The small model below shows the consequence of the figure's empty corner. It applies the common-agency result to the developer and is the book's illustration, not a result from the sources (inference).

The same structure is what Anton Korinek and Avital Balwit call the difference between direct alignment, with the goals of the entity operating a system, and social alignment, with its effects on society (Korinek and Balwit, 2023) (Section 14.4.1). Applied to the developer, a developer aligned with every party at the table can still be misaligned with everyone else (inference). Andrew Critch and David Krueger's 2020 preprint, which sorts the field's research by the number of humans and systems involved, from single/single to multi/multi delegation, is recalled from Section 14.4.3 (Critch and Krueger, 2020) (preprint).

Governments are principals with aims of their own. China's interim measures require public generative AI services to adhere to core socialist values, which Section 17.5 reports as a state writing part of a model's specification by law (Cyberspace Administration of China et al., 2023). The United States' 2025 action plan opens: "The United States is in a race to achieve global dominance in artificial intelligence (AI)" (House, 2025) (government document). A June 2026 order directs the design of a voluntary framework through which developers would give the government access to models designated by their cyber capabilities before releasing them more widely, which Section I.1.8 reports (Executive Office of the President, 2026). For the public this is a second delegation (inference). It reaches the developer through its government, an agent with aims and agency costs of its own, so the costs of the two delegations compound (Chapter 65; Chapter 66).

Staff are principals as well as agents. Matching the mission of an organization and its workers economizes on incentive pay, and attaches the workers' motivation to that mission rather than to the employer, as Section 14.5.1 derives (Besley and Ghatak, 2005; Akerlof and Kranton, 2005). The 2024 open letter in which current and former employees of frontier developers asked for protection when warning of risks is staff using voice, and Section 53.4.1 reports it with the statute that answered part of it (Current and companies, 2024). Two features make staff an unreliable channel for the public (inference). Their influence rests largely on the threat of leaving, which weakens as the mission they joined for changes, and their equity ties their wealth to the investors' interest.

Where the model stops: common agency describes principals who offer payment schedules, each with a fixed part they can adjust, to an agent who accepts the best combination (assumption A12 of Section 14.8.1). A government's instruments are laws that take years, and a public's are elections that bundle AI with everything else, so the model's timing fails (inference). The model also leaves out an agent that acts on its principals, by lobbying, by supplying the experts regulators rely on, or by setting the standards it will be judged by; Section 61.6 takes that up.

61.4 Professional norms and their limits #

One of society's answers to delegation when the client cannot judge the work even afterward is the profession, as in medicine, law, and auditing. Could the people who build or evaluate frontier models be governed that way? This section asks what the readable record says professions do, what they would require here, and where the analogy stops.

The founding works on professions, Andrew Abbott's The System of Professions and Eliot Freidson's Professionalism, the Third Logic, could be read here only as catalog records, as could Arrow's paper, so their arguments are not reported (Abbott, 1988; Freidson, 2001; Arrow, 1963). How professional schools instill norms, and what the readable record shows of it, is recalled from Section 25.6.

The readable economics places professional norms among the institutions that answer a credence good. "The problem of undertreatment is most famously controlled for by the Hippocratic Oath of a physician and its counterpart in the law," Dulleck and Kerschbamer write. They list it beside warranties, the separation of doctors from pharmacies, and the fixed part of a taxi tariff (Dulleck and Kerschbamer, 2006). They also state the oath's limit. In medicine strict liability is hard to impose, since a failed treatment is no sure sign of undertreatment. "With individual physicians, the Hippocratic Oath (and/or intrinsic motivation) may work reasonable well as a substitute for formal liability. With profit seeking institutions, this solution does not work" (Dulleck and Kerschbamer, 2006). The authors suggest that this may explain complaints that health maintenance organizations in the United States provide inexpensive treatment where expensive treatment is needed.

The AI ethics literature has drawn the same comparison from the other side. Brent Mittelstadt's argument that AI development lacks medicine's common aims and fiduciary duties, professional history and norms, methods for turning principles into practice, and accountability mechanisms is reported in Section 16.5.2 (Mittelstadt, 2019). Thilo Hagendorff's 2020 evaluation of AI ethics guidelines, reported in Section 2.5.1, also finds that AI ethics "lacks mechanisms to reinforce its own normative claims." Enforcement might involve reputational losses or "restrictions on memberships in certain professional bodies," he writes. "Yet altogether, these mechanisms are rather weak and pose no eminent threat" (Hagendorff, 2020). Elettra Bietti's 2020 paper, read in its abstract, names a failure on each side. The word "ethics" is "increasingly identified with technology companies' self-regulatory efforts and with shallow appearances of ethical behavior," while the condemnation of such "ethics washing" has bred an "ethics bashing" that trivializes moral philosophy (Bietti, 2020).

Two groups could become professions (inference). For evaluators the model is the audit profession, whose independence and failures are Section 38.6's, including John Coffee's question of why the gatekeeping professions did not bark (Coffee, 2006), and whose institutional design is Chapter 67's. For developers, a profession would need three things: a body of knowledge with agreed standards of practice, a credential that a professional body can withdraw, and discipline or liability that falls on individuals; these are the features the accounts above find missing or weak. As of October 2026 we found, in the sources gathered for this book, no licensing or professional body for the people who develop or evaluate frontier models. What exists is narrower. The California statute protects a "covered employee," defined by a role in assessing or managing risk, who reports a catastrophic danger to authorities inside or outside the developer (Section 53.4.1) (Legislature, 2025b). That protects an individual's voice without making the individual answerable to a profession.

The analogy stops at the firm (inference). Professions discipline individuals who serve individual clients. Frontier development is done by firms, release decisions are made by their leadership, and the people harmed are third parties; Dulleck and Kerschbamer's own limit, that an oath does not work for profit-seeking institutions, applies directly. And the gap in competence between practitioner and client is roughly fixed in medicine, while between a developer and the public it may grow with each model generation, which none of the credence-goods models describes.

61.5 Corporate forms and charters as commitment devices #

Developers have tried to bind themselves through how they are incorporated and governed. Whether those structures bind depends on a mechanism that comes before any of the cases, so this section teaches the mechanism first, then asks what each structure commits to and who can enforce it. The theory of boards and the structures themselves are reported in Section 53.6; this section reads them as commitments.

61.5.1 What makes a commitment bind #

A commitment is a choice made now that changes what a later choice costs. The readable literature defines the failure it addresses. James Fearon, writing on why rational states go to war, describes commitment problems as "situations in which mutually preferable bargains are unattainable because one or more states would have an incentive to renege on the terms." He notes that anarchy, understood as the absence of an authority capable of policing agreements, is routinely cited as a cause of war (Fearon, 1995). Gailmard and Patty's review recalls an earlier wave of models that treated bureaucracy itself as a device for making policy commitments last (Gailmard and Patty, 2012). Three founding works could be read only as catalog records, so their content is not reported. They are Jon Elster's Ulysses and the Sirens, Thomas Schelling's The Strategy of Conflict, and Finn Kydland and Edward Prescott's paper whose title states the problem, "Rules Rather than Discretion: The Inconsistency of Optimal Plans" (Elster, 1979; Schelling, 1960; Kydland and Prescott, 1977). The derivation below is the book's own.

The derivation turns the question of whether a structure binds into two questions that can be asked of any document (inference). Who can impose the cost of breaking it, and what does it cost to rewrite it first?

61.5.2 What the structures commit to #

From law. Law's general category for an agent entrusted with power over another's interests is the fiduciary, and Tamar Frankel's account finds that all fiduciary relations "pose the problem of abuse of delegated power" (Section 19.3) (Frankel, 1983). Frankel's account of the trust names the party who enforces it: the settlor delegates power to the trustee, and "the beneficiary merely obtains rights by law to enforce the will of the settlor against the trustee" (Frankel, 1983). In the default corporation that party is the stockholder, as the developer's description in Section 61.3 states (Anthropic, 2023f).

Anthropic's Long-Term Benefit Trust is the most fully described structure. Section 53.6.1 reports its trustees, the public benefit form it pairs with, and the developer's concession that the form does not make directors accountable to other stakeholders. Its terms as commitments are these, all in the developer's own account (Anthropic, 2023f) (developer document). A class of stock held only by the trust gives it the authority to elect and remove board members, phasing in "according to time- and funding-based milestones," and the 2023 post states that the trust will elect a majority of the board within four years. "Protective provisions" require that the trust receive notice of certain actions that could significantly alter the corporation. The trust is a "purpose trust" under Delaware law, whose purpose is the developer's own. Trustees serve one-year terms, and future trustees are elected by the trustees. On amendment, the developer expects most changes to be made by agreement of the trustees with the board or with the other stockholders. It has also designed "failsafe" provisions that "allow changes to the Trust and its powers without the consent of the Trustees if sufficiently large supermajorities of the stockholders agree," and the required supermajorities rise as the trust's power phases in (Anthropic, 2023f).

Read with the derivation, the stockholder supermajority is the trust's $c_{\text{amd}}$, and it is held by the stockholders, the party whose interest the commitment restrains (inference). The developer states the trade-off itself: "It's important to prevent this arrangement from being easily undone, but it is also rare to get something like this right on the first try" (Anthropic, 2023f). The post does not give the supermajority thresholds, so the size of $c_{\text{amd}}$ cannot be read from the public record. The protective provisions require notice, not consent, and the account does not say who could enforce the trust's purpose against the trustees (inference).

OpenAI's 2018 charter makes two commitments of a different form (OpenAI, 2018) (developer document). It states: "Our primary fiduciary duty is to humanity." It also commits that if "a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project." Specifics are to be worked out "in case-by-case agreements," with a typical trigger of "a better-than-even chance of success in the next two years" (OpenAI, 2018). Read with the derivation and with Frankel's account, the charter names humanity as the party owed the duty but no party holding rights to enforce it (inference). The stop-and-assist clause leaves both the trigger and the terms to the committing party, so $c_{\text{ren}}$ is what the developer chooses to bear (inference). The developer's 2025 recapitalization, which it says followed nearly a year of dialogue with the offices of the attorneys general of California and Delaware, is reported in Section 53.6.1 (OpenAI, 2025a). That section finds that the sources read here do not establish who outside the structure can enforce the form.

The theory of what such structures achieve is thin in the readable record. Henry Hansmann's 1980 paper on the role of nonprofit enterprise could be read only as a catalog record, so its prediction is not stated here (Hansmann, 1980). Lucian Bebchuk and Roberto Tallarita's argument, in a 2020 preprint, that corporate leaders have strong incentives not to protect stakeholders beyond what serves shareholder value, so that protection needs outside intervention, is reported in Section 53.6 (Bebchuk and Tallarita, 2020) (preprint). A further proposal has the same form. The Windfall Clause, a 2019 preprint by Cullen O'Keefe and colleagues with affiliations at the Centre for the Governance of AI, is "an ex ante commitment by AI firms to donate a significant amount of any eventual extremely large profits" (O'Keefe et al., 2019) (preprint). Its authors write that an enforceable clause would be "a credible ex ante commitment," and they worry that it "could be circumvented" by legal means, such as "contractual loopholes or strategic accounting practices," or by extralegal ones (O'Keefe et al., 2019). In the derivation's terms, a clause whose $\Pi$ is by design enormous needs a $c_{\text{ren}}$ larger still, and the authors' worry is that the real $c_{\text{ren}}$ is smaller than the clause's text suggests (inference).

61.5.3 Statutes as enforcers #

A statute can supply the outside party the derivation asks for. As of October 2026 California's 2025 law on large frontier developers requires such a developer to "write, implement, comply with, and clearly and conspicuously publish" a frontier AI framework (Legislature, 2025b). Failing to comply with it brings a civil penalty capped per violation and recoverable only by the Attorney General, materially false or misleading statements about compliance are barred unless made in good faith and reasonable under the circumstances, and a material modification must be published with a justification (Section I.1.12) (Legislature, 2025b).

Read with the derivation, the statute raises $c_{\text{ren}}$ for breaking the framework in force, by a sum capped per violation, and sets $c_{\text{amd}}$ to the cost of publishing a revision with a reason (inference). It thus turns silent reneging into public revision. The commitment then binds as far as a published revision is costly, in reputation and in the attention of regulators, which is why the revision record is the evidence Section 62.4 reads. For scale, the statute defines a large frontier developer as one that, with its affiliates, had annual gross revenues above five hundred million dollars in the preceding calendar year (Legislature, 2025b), so the cap per violation is 0.2 percent of that revenue threshold. How many violations one breach counts as, the text read here does not settle (inference). The statute as a whole is Section 63.2' subject, and Section 62.5 reads it as enforced self-regulation.

Where the analysis stops: as far as the sources gathered show, none of these commitments has been tested in public against a large commercial cost with the record open to outside review. The section can therefore report designs and the predictions the derivation makes, not outcomes. Section 53.6.2 states the observable that would decide between the readings, whether a commitment holds on an occasion when keeping it costs money. The derivation adds what to look for on that occasion: which party imposed the cost, and whether the commitment was rewritten first (inference).

61.6 Capture, legitimacy, and power #

The relationship invites three objections. The monitors may come to serve the developer; a private body writing rules for everyone may lack the standing to do so; and a developer that aligns its models well may thereby become too powerful. This section states each with the evidence that bears on it.

61.6.1 Capture #

Capture is the condition in which regulation serves the industry it regulates. George Stigler's 1971 theory, by its abstract, analyzes the uses of public resources and powers to improve the economic status of groups such as industries, as the demand for regulation. It then sketches the features of the political process that let "relatively small groups" obtain it, as its supply (Stigler, 1971) (Section 1.3). Jean-Jacques Laffont and Jean Tirole's 1991 paper in the Quarterly Journal of Economics develops, by its abstract, "an agency-theoretic approach to interest-group politics." Among its results, the organizational response to the possibility of capture is to reduce the stakes interest groups have in regulation. Another is that "an interest group has more power when its interest lies in inefficient rather than efficient regulation, where inefficiency is measured by the degree of informational asymmetry between the regulated industry and the political principal" (Laffont and Tirole, 1991). An edited volume of 2013, by its abstract, argues that capture "is often misdiagnosed and may in fact be preventable and manageable," and advances "a more rigorous and empirical standard for diagnosing and measuring capture" (Cambridge University Press, 2013). Daniel Carpenter's study of how agencies forged their autonomy and Barry Weingast and Mark Moran's paper on congressional control of the Federal Trade Commission could be read only as catalog records (Carpenter, 2002; Weingast and Moran, 1983).

By the measures of Section 61.1.2, the informational asymmetry between frontier developers and legislatures is large, which is where Laffont and Tirole's abstract places an interest group's power (inference). Developers also supply much of what a regulator or evaluator needs, the models, the data about them, and many of the people who understand them, which is the expertise channel of capture; Section 63.3 states it as a hypothesis to measure. The same dependence reaches the evidence base, as Section 5.6.4 reports. Industry's share of AI researchers, compute, and leading models has grown (Ahmed et al., 2023). Large technology companies directly fund computer science faculty, including those who publish on ethics and fairness (Abdalla and Abdalla, 2021). An essay's title names the steep cost of capture (Whittaker, 2021), and a paper examines corporate capture at a fairness conference (Young et al., 2022).

61.6.2 Whose safety #

A second objection holds that the safety framing itself serves the developers. Seth Lazar and Alondra Nelson argue that experts, technologists especially, should not decide alone what risks count (Lazar and Nelson, 2023). Timnit Gebru and Émile Torres trace the normative framework behind the goal of artificial general intelligence, by their abstract, to the Anglo-American eugenics tradition of the twentieth century (Gebru and Torres, 2024). Both are reported in Section 2.5.2 and weighed in Section 4.4.4, together with an outside study of the safety field's epistemic culture (Ahmed et al., 2024). Yoshua Bengio, Geoffrey Hinton, and colleagues, senior researchers writing in Science in 2024, make a claim from the other direction. "Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems," they write, and "We urgently need national institutions and international governance to enforce standards preventing recklessness and misuse" (Bengio et al., 2024b).

Read against this chapter's ledger, the two positions disagree about which harms count in the residual loss, not about whether the developer needs a monitor it does not control (inference). The measurements of industry's concentration and funding ties, which concern AI research and AI ethics rather than alignment research (Section 5.6.4), fit both. They support the critique's worry about who shapes the questions, and they equally support the consensus statement's call for outside institutions, since an evidence base at risk of capture is a reason for monitors independent of the developers. The critique's strongest reply is that monitors chosen and briefed within the safety framing would inherit its narrowness, so independence from the developers does not by itself answer it (inference). The book's answer to the framing objection is structural: by stating the developer as the agent, this part makes "who chose this specification" a question it asks of every framework, rather than one it leaves to critics (inference). The opportunity-cost argument and its evidence are Section 4.4's.

61.6.3 Legitimacy and non-domination #

Philip Pettit's republican account is reported from its abstracts in Section 20.4.5 and Section 17.5. It defends freedom as non-domination, "an alternative, both to the negative view of freedom as non-interference and to the positive view of freedom as self-mastery" (Pettit, 1997). It holds that state coercion does not subject people to domination "insofar as we enjoy an equally shared form of control over those in power" (Pettit, 2012). Section 21.7.2 applies the second abstract to a model; its application to the public belongs here.

Applied to the developer, the two tests come apart (inference). On a test of non-interference, a developer raises a problem only when its decisions harm people. On Pettit's condition for power without domination, what matters is whether the people affected share control over those who hold the power, and a developer over which the public shares no control fails it however the developer decides. This is the sense in which an unchecked developer is a problem however benevolent, a gloss that is the book's and not Pettit's. Where the argument stops: Pettit states the condition for the coercion a state exercises through law, and a developer coerces no one by law. Its power is over which technology exists and how it behaves, and whether that is the kind of power the condition covers, the abstracts do not say (inference). Participation by the public in a developer's specification, and the case against it in a 2023 paper titled "Against 'Democratizing AI'" (Himmelreich, 2023), are Section 17.7's.

61.6.4 Power #

A system that does exactly what its controller wants multiplies its controller's power. Tom Davidson, Lukas Finnveden, and Rose Hadshar's 2025 argument that systems singularly or secretly loyal to a small group are routes to seizing power, once AI is far more capable than it was then and deployed in institutions such as governments and militaries, is reported in Section 38.5.1 (Davidson et al., 2025) (not peer reviewed). The audits that would catch a loyalty inserted by a developer are Section 38.5's, the human insiders who could insert one are Section 53.5's, and the consequences for the distribution of power are Chapter 68's. Their place in this chapter is one inference: the more reliably a developer can align its models to itself, the more the public's problem becomes aligning the developer (inference). Releasing models openly does not by itself answer this: David Gray Widder, Sarah Myers West, and Meredith Whittaker find, in a 2024 Perspective in Nature, that "openness alone does not perturb the concentration of power in AI" (Widder et al., 2024). Release rules are Section 63.7's.

Outside ratings are one way the public checks a developer without access to it. An expert panel convened by the Future of Life Institute, an advocacy organization, grades developers' safety practices from public documents and a survey, and Section I.5.4 reports the grades (Future of Life Institute, 2025). They are evidence of a different kind from an audit: a panel's judgment, by an organization with a position on the question, of what developers chose to publish.

61.7 What the relationship means for delegation #

The laboratory as the public's agent is where the book's thesis fits most directly. Of the assumptions in Section 14.8.1, the ones that fail for a model hold here: the developer's aims exist before the public acts, it can refuse any arrangement offered, and no one can rerun it. The classical tools therefore apply without the intentional stance, the method of describing a system as wanting and believing in order to predict it (Section 1.5), and this chapter's uses of them are applications, not analogies (inference). The agency-cost ledger organizes the part. The credence-goods model predicts undertreatment and overcharging when liability and verifiability are both weak. Common agency predicts an equilibrium that serves the parties at the table. And the commitment derivation predicts that a self-written commitment binds only as far as someone else can make breaking or rewriting it costly.

What does not carry over is as informative (inference). The firm's mechanism for moving agency costs onto the agent, a market that prices them, has no counterpart for the public. The expert in the credence model knows the customer's problem, while a developer may not know its model's. The competence gap between developer and public may grow with each generation, which no model in this chapter describes. And the agent acts on its principals, by supplying regulators and evaluators with their tools and people, in ways the delegation models leave out. These are the points at which the institutions in the rest of this part must do more than their precedents did.

Every claim here concerns the third relationship, with one link to the second: a developer's report about its own model is evidence about the deployed system and also an act of the agent the public is trying to oversee (Section 5.6). None of the five results on the negative-space list of Section 1.7.1 comes from this relationship. Its novelty, the agent's power over the means of checking it, strains the old tools rather than breaking with them (inference).

61.8 Settled, contested, missing #

61.9 Exercises #

Further reading #

• Jensen and Meckling (1976): agency costs as monitoring, bonding, and residual loss, and why the agent bears them when a market prices them.
• Dulleck and Kerschbamer (2006): when experts exploit what clients cannot verify, and which institutions stop them.
• Dixit et al. (1997): what an agent serving many principals does, and whom it leaves out.
• Stigler (1971): the demand for regulation by economic groups and the political process that supplies it to small groups, read here in its abstract.
• Pettit (1997): freedom as non-domination, set against freedom as non-interference, read here in its abstract.
• Ahmed et al. (2023): the measurement of industry's growing share of AI researchers, compute, and leading models.