Draft complete but not yet reviewed; expect revisions

Competition and Races

The fourth of the book's problems is keeping alignment true under pressure, and the pressure most often named is competition. A laboratory that spends an extra month on evaluation, declines to ship a capability, or trains a smaller model to keep it checkable pays a cost that its rivals do not; a state that slows its developers pays it in national terms. If the prize goes to whoever arrives first, each party has a reason to cut the time and money that safety takes, even when every party would prefer that all of them took it. This chapter is about that reason: whether it holds, how strong it is, and what has made rivals hold back before.

The chapter concerns the third of the book's relationships, the developer and the public, and the relationship between states, rather than any model: it applies the four problems one level up (Chapter 61), with competition as the force acting on maintenance. Two other relationships enter at the edges. A commercial signal can reach a model's training, and a principal under pressure chooses between a more capable system and a more trusted one. Economists and game theorists have modeled races and arms races for decades, and their models say under what conditions competition should erode care. The chapter teaches those models first, then turns to the evidence, which is thinner than the models. It gives equal space to the case that a decisive race exists and to the case that the race is a frame, chosen by the people who invoke it, that makes itself true. It ends with the conditions under which developers and states have held back, which is where the pause debate belongs.

65.1 What a race is, and whose problem it is #

A common claim is that competition forces developers to trade safety for speed. The claim bundles three different situations, and the models of the next section apply to only two of them.

The first is ordinary product competition. Firms compete for customers on price and quality, and safety can be part of quality. In a 2019 preprint, three researchers then at OpenAI, Amanda Askell, Miles Brundage, and Gillian Hadfield, argue that safer systems "will tend to have a higher market value" and are less likely to cause accidents for which their makers are held liable (Askell et al., 2019). Where that holds, competition rewards care.

The second is a race: a contest in which the first to reach a goal takes most of the reward. In a winner-take-all race the others get nothing; in a winner-take-most race they get less. Speed is then valuable in itself, and anything that slows a team costs it a share of the prize. This chapter calls that cost safety investment: the time, compute (the computing power used to train and run models), staff, and forgone capability a developer spends on reducing risk. Askell and colleagues argue that "even with these incentives in place," competitive pressure can lead companies to invest less in responsible development than they otherwise would (Askell et al., 2019). The resulting race to the bottom is, in their words, a collective action problem: "a situation in which all agents would be better off if they could all cooperate with one another, but each agent believes it is in their interest to defect rather than cooperate" (Askell et al., 2019).

The third is an arms race between states. Each state arms because the others do, no authority above them can enforce an agreement, and the stake is security rather than market share. Its theory is the subject of Section 65.3.

Product competition and the race between firms concern the developer and the public. The arms race concerns states, and each state's relationship to the public it answers to. Which of the three describes the development of frontier AI, the most capable systems of the moment, decides whether the race models apply at all. Section 65.5 sets out the evidence on both sides.

65.1.1 The four problems one level up #

Section 61.2 applies the book's four problems to the laboratory as the public's agent, and competition acts on each of them (inference). It acts on specification, what a developer chooses to optimize: a cheap signal of what users liked entered one developer's training, as Section 65.4 recalls. It acts on instillation, the developer's safety culture (Chapter 53), and on verification, what it discloses (Chapter 67). And it acts on maintenance. The derivation in Section 61.5.1 shows that a commitment binds only while the gain from breaking it stays below the cost of breaking or rewriting it. A commitment written when stakes were small therefore stops binding first where competition makes the gain largest.

65.1.2 States are principals too #

A developer serves several principals, parties on whose behalf an agent acts, at once; Section 14.4 models the situation as common agency: investors, users, the public, and governments, each pulling with its own instruments. A government that frames AI as a contest to win sets the incentives of its developers. The United States' AI Action Plan of July 2025 is titled "Winning the Race," and its opening quotation from the President states that "it is a national security imperative for the United States to achieve and maintain unquestioned and unchallenged global technological dominance" (House, 2025) (government document). A December 2025 executive order, which Section 63.3.5 reports, states that "to win, United States AI companies must be free to innovate without cumbersome regulation" (Executive Office of the President, 2025a).

A government can also act on the race directly. In October 2022 the US Commerce Department's Bureau of Industry and Security imposed export controls on advanced computing chips and on certain semiconductor manufacturing items, citing their role in China's military modernization (Bureau of Industry and Security, 2022). Its January 2025 framework for AI diffusion revised those controls and added a control on the weights, the learned parameters, of certain advanced models whose weights are not published (Bureau of Industry and Security, 2025). Other states write requirements of their own: China's 2023 interim measures for generative AI services, which Section 17.5 reads, specify part of what a model must and must not say (Cyberspace Administration of China et al., 2023). A developer that answers to a state racing another state is an agent whose principal has its own reasons to want speed. For the public that is a second delegation, and Section 61.3 notes that the costs of the two compound.

65.1.3 Words with owners #

"AI race" is a term with owners, used with opposite intent. Governments use it to organize policy around winning, as the title of the action plan shows (House, 2025). Critics use it to name a hazard: Stephen Cave and Seán ÓhÉigeartaigh's 2018 paper, by its abstract, assesses the risks of "the AI race narrative" as well as of an actual race, such as incentivizing corner-cutting on safety and increasing the risk of conflict (Cave and ÓhÉigeartaigh, 2018). This chapter uses "race" only in the technical sense above and reports each party's usage as theirs.

The chapter's subject is also what Section 4.1.1 calls structural risk: harm that comes from how a technology reshapes incentives and institutions, so that changing one agent's behavior would leave the level of risk largely unchanged (Zwetsloot and Dafoe, 2019). Allan Dafoe's chapter on AI governance, by its abstract, singles out great power security competition as a particular challenge, "among the hardest dynamics to govern" (Dafoe, 2023). A race is the clearest case: no single developer's restraint removes it.

65.2 Race models in economics and game theory #

Under what conditions does a race erode care, and how much? Two literatures answer before any claim about AI is made. Economics asks why firms race and why markets under-supply some kinds of knowledge. Game theory builds models in which teams choose how much safety to trade for speed, and solves for what they choose. This section takes them in turn, derives the central result of the founding AI race model, and ends with what all the models leave out.

65.2.1 Why markets race #

From economics. Economists have modeled patent races, contests among firms to make an invention first and own it, since at least the late 1970s. The founding papers, by Glenn Loury, by Partha Dasgupta and Joseph Stiglitz, and by Christopher Harris and John Vickers, could be read here only as catalog records, as could the paper by Philippe Aghion and colleagues on competition and innovation (Loury, 1979; Dasgupta and Stiglitz, 1980; Harris and Vickers, 1987; Aghion et al., 2005). This chapter reports their findings only as readable sources carry them.

Askell and colleagues suggest that an AI development race may resemble a patent race. They report Dasgupta and Stiglitz's argument that a monopolist will outspend its rivals on research to prevent a duopoly, and note that the literature on research races, Harris and Vickers among it, holds that competition is more intense among close competitors (Askell et al., 2019). Nick Bostrom's 2017 paper on openness in AI development carries a fourth claim, which he attributes to Kenneth Arrow among others: basic ideas are public goods, and without some market power a firm "is unable to appropriate the value of the new ideas it originates" (Bostrom, 2017; Arrow, 1962).

Two implications follow for safety (inference). The private value of arriving first includes the share of the prize taken from rivals, which is a transfer between firms rather than a gain to anyone else, so a firm's incentive to buy speed can exceed the value of that speed to the public. And safety knowledge is the clearest case of an idea its originator cannot appropriate: a developer that publishes a method for catching a failure pays for it alone and shares its benefit with every rival. Together these say where competitive pressure on care should be strongest, among close rivals racing for a large prize, rather than that it is strong everywhere.

The analogy stops at the finish line. A patent race ends when the patent is granted, and the loser keeps what it had. A frontier capability lead may compound if AI comes to automate AI research (Chapter 55), and the loser of an AI race may also bear the winner's accident. The first difference is a question of fact that Section 65.5 takes up; the second is what the AI race models add.

65.2.2 The founding AI race model #

From game theory. The earliest of the AI race models in Table 65.1, and one that three of the later papers cite, is by Stuart Armstrong, Nick Bostrom, and Carl Shulman. It circulated as a technical report of the Future of Humanity Institute in 2013 and was published in AI & Society in 2016; the copy read here is the report (Armstrong et al., 2016). Several teams compete to build the first AI. Each team has a level of capability and chooses a level of safety precautions, from total safety to none. Its score is its capability minus its safety, and the team with the highest score finishes first. The winner's project then succeeds with probability equal to its safety, and otherwise causes what the authors call an AI-disaster. A team values its own success at 1 and a disaster at 0. Enmity sets how much it minds a rival's success: a rival's success is worth $1 - \epsilon$ to it, from indifference ($\epsilon = 0$) to regarding a rival's success as no better than a disaster ($\epsilon = 1$) (Armstrong et al., 2016).

Capabilities are drawn independently and uniformly from zero to a maximum. A high maximum means, in the authors' words, that capability is the dominant factor, so that "one can achieve very little extra by skimping on precautions"; a low maximum means risk-taking buys a large chance of winning (Armstrong et al., 2016). The teams play a Nash equilibrium, the outcome no participant wants to leave on its own (Section 57.2.1), which the authors suggest could arise from each team cutting its safety a little at a time in response to the others. They solve three cases: no team knows any capability, each knows only its own, and all know everyone's (Armstrong et al., 2016). The derivation below works the first and the last, with $n_{\text{team}}$ teams, capabilities $\chi_i$ uniform on $[0, \bar\chi]$, a safety level $p_{\text{safe}}$, and the enmity $\epsilon$; the letters are the book's, not the paper's.

Equation (65.1) says that a leader everyone can see takes full safety only once its lead reaches the enmity, and takes none when the race is neck and neck. Equation (65.2) says that in ignorance, safety falls with enmity and with the number of teams and rises with the importance of capability. With no enmity at all, both give full safety: a team that does not mind who wins has no reason to race. More teams also make close races more likely, since the expected lead of the top team over the second is $\bar\chi / (n_{\text{team}} + 1)$ for uniform capabilities.

The paper reports the overall chance of disaster in each case, and Figure 65.1 recomputes it from the paper's closed forms. The chance falls as capability matters more; near a spread of zero the only equilibrium is no precautions at all; and lowering enmity always lowers it. Adding teams raises it with no information and with full information. With private information it does so only when capability matters little, because only teams of low capability take risks (Armstrong et al., 2016). The surprise is information. No information is always the safest case. At maximal enmity the authors find private information safer than public, but for low spread and low enmity public information can be safer; at the figure's defaults, halving the enmity reverses the order. The authors call this an information hazard: "we'd be better off not knowing" (Armstrong et al., 2016). Their remedies, in Table 65.1, follow from the parameters.

The model's assumptions bound what it shows. Speed comes only from cutting safety, only the winner's safety matters, and the teams reach equilibrium. It is a theorem about a stated game, and it is evidence about the world only to the degree that the game describes the world.

65.2.3 Decisiveness, evolution, and cooperation #

Later work changes the founding model's assumptions. Nicholas Emery-Xu, Andrew Park, and Robert Trager's 2024 paper in the Journal of Conflict Resolution models states racing to field "a powerful, dangerous new military technology" that may cause a disaster affecting all states; it is read here from its abstract (Emery-Xu et al., 2024). By the abstract, more decisive races, "in which small performance leads produce larger probabilities of victory," are usually more dangerous. Revealing rivals' capabilities has two opposing effects: states learn either that they are far apart and compete less, or that they are close and "drastically reduce safety to win." The public information case is therefore less risky than the private one "except under high decisiveness." And the larger the eventual loser's impact on safety relative to the winner's, the more dangerous the race (Emery-Xu et al., 2024). The information result of the founding model thus depends on how decisive the race is, and the loser's share of the risk, which the founding model sets to zero, matters (inference).

The Anh Han and colleagues, in a 2020 paper in the Journal of Artificial Intelligence Research, use evolutionary game theory, in which strategies spread through a population by imitation of the more successful (Han et al., 2020). In each round of a repeated race a developer plays safe, paying a cost, or unsafe, moving faster but risking disaster and being found out. When the prize for winning far outweighs what a developer earns in each round, always playing unsafe dominates unless the risk of disaster is high; when the prize lies far off, the safe strategies take over unless both the chance of being found out and the risk of disaster are low. Only in a "dilemma zone" of intermediate risk do individual and collective interest diverge, and the authors argue that regulation helps only there; elsewhere it has no effect or does harm, and in a long race it may be risk-taking that needs encouragement (Han et al., 2020). They list their own simplifications: speed comes only from cutting safety corners, safe play never causes a disaster, and the rhetoric of the race is left for future work (Han et al., 2020). A sequel by Han and colleagues, whose title states that voluntary safety commitments provide an escape from over-regulation, is cited here by its title only (Han et al., 2022).

Askell and colleagues write the race as a two-player game of cooperating, by keeping an acceptable level of investment in responsible development, or defecting, by cutting it (Askell et al., 2019). From the payoffs they derive five factors that make cooperation likelier, which Section 65.3 connects to a threshold of trust. They add that close frontrunners would likely worsen the problem, consistent with the founding model (Askell et al., 2019). Table 65.1 sets the models side by side, with two that the next sections report.

65.2.4 What the models share, and what they leave out #

Every model in the table treats safety as one quantity traded against speed, and none lets safety work be the better investment for speed: Askell and colleagues assume only that responsible development contributes less to development speed than other investments, "not that they contribute nothing" (Askell et al., 2019). Han and colleagues state both at once as their first simplification, that higher speed comes only from cutting safety corners (Han et al., 2020). Neither assumption fits the trade-offs developers describe (inference). A developer's safety spending is evaluation time, deployment delay, staff, and compute for alignment research, each with its own cost in speed, and a method such as learning from human feedback can make a model both safer and more useful, as Section 65.4 recalls. A model in which safety work is sometimes the better investment could reverse the sign of the incentive for that work, and as of October 2026 we found none. The models say which features of a race erode safety: closeness, enmity, decisiveness, and the number of rivals. Whether the AI contest has those features is a question of fact.

65.3 Why rival states arm, and when reassurance works #

Firms race inside a state that can enforce their contracts and punish their collusion. States race with no authority above them. This section teaches the theory of that situation: why states arm when all would prefer that none did, why rational rivals fail to strike the bargains that would leave both better off, and the distinction that decides whether reassurance can work at all.

65.3.1 The security dilemma #

From international relations. John Herz named the security dilemma in 1950, for groups that live side by side with no higher authority. Seeking security from attack, by the passage the record gives as his abstract, they "are driven to acquire more and more power in order to escape the impact of the power of others. This, in turn, renders the others more insecure and compels them to prepare for the worst" (Herz, 1950). Robert Jervis's 1978 paper, read here from its abstract, asks when the dilemma bites less. It operates less strongly, he finds, "when the defense has the advantage over the offense and when defensive postures differ from offensive ones," two variables that geography and technology shape (Jervis, 1978). Charles Glaser's review of arms races, by its abstract, separates a state that arms because arming is its best option, whose race "has no consequences of its own," from one whose arming domestic interests have distorted (Glaser, 2000). Jervis's book Perception and Misperception in International Politics, in its 2017 edition, could be read here only as a catalog record (Jervis, 2017).

Jervis's two questions can be asked of AI. The first is whether capability favors attack as it scales. Section 58.2.2 reports the paper by Garfinkel and Dafoe whose simple models of invasions and cyberattacks suggest that growing investment favors offense at low levels and defense at high ones (Garfinkel and Dafoe, 2019). The second is whether one party's safety work can be told apart from capability work by a rival watching from outside. Section 5.2.2 reports that many safety benchmarks correlate highly with capability, so even a measurement may not separate them (Ren et al., 2024b). Ben Buchanan's book on the cybersecurity dilemma, which Section 58.8 reports from its abstract, adds that states that do not seek to harm each other still find it prudent to penetrate each other's networks. A state's means of securing itself then threatens the security of others (Buchanan, 2017). On both of Jervis's variables, then, the evidence available for AI leaves the dilemma severe or undetermined (inference).

65.3.2 Prisoner's dilemma or trust dilemma #

Whether reassurance can help depends on the shape of the game. Section 57.2.1 teaches the prisoner's dilemma. Write its payoffs for a race: $\mathrm{T}$ to a state that races while its rival restrains, $\mathrm{R}$ to each when both restrain, $\mathrm{P}$ to each when both race, and $\mathrm{S}$ to a state that restrains while its rival races. In a prisoner's dilemma, $\mathrm{T} > \mathrm{R} > \mathrm{P} > \mathrm{S}$, and racing is better whatever the rival does. In a trust dilemma, also called a stag hunt, both prefer mutual restraint to winning alone, $\mathrm{R} > \mathrm{T}$, but each still prefers racing to being the only one who held back, $\mathrm{P} > \mathrm{S}$. Corin Katzke and Gideon Futerman define it this way: "both players would rather cooperate than defect; however, if one player defects, it is better for the other to defect as well" (Katzke and Futerman, 2024). The letters keep the upright form of Section 57.2.1; the derivation below shows what separates the two games.

The threshold is reassurance theory in one line (inference). Verification, transparency, and dialogue act on the belief and leave the payoffs alone. In a trust dilemma they work once they carry the belief past $p_{\text{res}}^\ast$; in a prisoner's dilemma they cannot work at all, and only changing the payoffs helps: a commitment that makes racing costly, an enforcer, or the repeated play of Section 57.2.2. The threshold falls as each term moves the way Jervis's abstract lists in words: cooperation is more likely "when the costs of being exploited and the gains of exploiting others are low, when the gains from mutual cooperation and the costs of mutual noncooperation are high, and when each side expects the other to cooperate" (Jervis, 1978). Askell and colleagues' five factors are the same terms: trust is $p_{\text{res}}$, shared upside a larger $\mathrm{R}$, low exposure a less costly $\mathrm{S}$, low advantage a smaller $\mathrm{T}$, and shared downside a worse $\mathrm{P}$ (Askell et al., 2019).

65.3.3 The case that the race to superintelligence is a trust dilemma #

Katzke and Futerman, in a December 2024 preprint, argue that a race between the United States and China to build superintelligence, "an AI system that vastly exceeds human performance across nearly all cognitive tasks," is a trust dilemma (Katzke and Futerman, 2024). Their argument is an internal critique. Two assumptions motivate racing: that superintelligence would give its first developer a decisive military advantage, and that states are rational actors that put survival first. The authors argue that the same assumptions make the race dangerous to the racer. A rival facing decisive disadvantage has reason to attack the project before it is finished; a decisive advantage requires the very rapid leap in capability where loss of control is most likely; and a controlled system would concentrate power at home (Katzke and Futerman, 2024). Under these dangers, if one state believes the other will not build superintelligence, "then that state will not develop ASI itself," and verification need not be perfect, because the critical probability falls as the gap between the two equilibria widens (Katzke and Futerman, 2024).

The argument turns on one payoff assumption, $\mathrm{R} > \mathrm{T}$: that winning the race, net of the risks of preemptive attack, loss of control, and concentrated power at home, is worth less than mutual restraint (inference). The authors' own example shows the opposite case: a realist state that gives itself a 90 percent chance of controlling the system, and a 20 percent chance that its rival builds one if it does not, is predicted to build despite the existential risk (Katzke and Futerman, 2024). They also do not take a position on how likely loss of control is (Katzke and Futerman, 2024). If the race could be won quietly, with a controllable system and a rival unable to respond, $\mathrm{T}$ would exceed $\mathrm{R}$ and the game would be a prisoner's dilemma again.

The rival view rests on the founding model's other branch. Leopold Aschenbrenner, whose 2024 essay Section 2.6.4 recalls, writes that "a healthy lead will be the necessary buffer that gives us margin to get AI safety right, too" (Aschenbrenner, 2024). That is equation (65.1) read as policy: a leader far ahead can afford safety. Katzke and Futerman argue against the buffer: a rival will hold off from attacking only if it believes it can still win, which requires a close race in which "the safety-buffer" would be "completely absent" (Katzke and Futerman, 2024). Each position needs a premise about leads. Aschenbrenner's needs a lead that is large, visible, and durable; Katzke and Futerman's needs a rival that sees the lead and can act on it. Which premise holds is the question of Section 65.5 (inference).

65.3.4 Why rational rivals fail to agree #

War and arms races destroy value, so why do rational rivals not strike the bargain that would leave both better off? James Fearon's 1995 answer proposes three mechanisms (Fearon, 1995). In the first, leaders know things about their own capabilities and resolve that rivals do not, and "they can have incentives to misrepresent such private information" to get a better deal. The second is the commitment problem, which Section 61.5.1 reports: a bargain both prefer fails because one side would later have an incentive to renege. Fearon argues that preventive war "stems from a commitment problem rather than from differential power growth per se." The third, issues that cannot be divided, he finds less compelling than the first two, "although it is logically tenable" (Fearon, 1995). Robert Powell, in a paper whose readable copy is a 2004 manuscript, argues that indivisibilities are really commitment problems, and that in preventive war, preemptive attack, and bargaining over future power alike, "large, rapid shifts in the distribution of power lead to war" (Powell, 2006). Thomas Schelling's The Strategy of Conflict, the older treatment of bargaining and commitment, could be read here only from its publisher's page (Schelling, 1960).

Both mechanisms apply to AI (inference). Developers and states hide what their systems can do, and the race models show that what each learns about the others changes how much safety it keeps. And the scenario in which AI automates AI research is a large, rapid shift in power by construction. A state that expects its rival to gain a decisive lead cannot trust any promise the rival makes about using it, which is the logic of preemption in Katzke and Futerman's first danger.

65.3.5 Deterrence by sabotage #

Dan Hendrycks, Eric Schmidt, and Alexandr Wang propose a standoff instead of an agreement. Their 2025 preprint introduces "Mutual Assured AI Malfunction (MAIM): a deterrence regime resembling nuclear mutual assured destruction (MAD) where any state's aggressive bid for unilateral AI dominance is met with preventive sabotage by rivals" (Hendrycks et al., 2025). The name is the authors' coinage, modeled on the nuclear standoff in which each side could destroy the other even after being attacked first. They argue that sabotage, from covert cyberattacks to strikes on datacenters, is easy enough that MAIM "already describes the strategic picture AI superpowers find themselves in." To keep it stable they propose clear escalation ladders, datacenters far from cities, and transparency that distinguishes destabilizing projects from acceptable use (Hendrycks et al., 2025). They also warn that competitive pressure could lead officials to tolerate a double-digit risk of losing control rather than fall behind (Hendrycks et al., 2025).

The security dilemma's criteria test the proposal (inference). Jervis's abstract says the dilemma is milder when defense has the advantage (Jervis, 1978), and MAIM rests on the reverse, a standing advantage of sabotage over protection. A regime built on offense is stable only while each side can tell the other's destabilizing project from ordinary AI work and can read each act of sabotage correctly, which is why the authors' own maintenance measures are about distinguishing and signaling. The espionage that tells a state when to sabotage is itself the kind of intrusion that, in Buchanan's account, threatens the security of the state it watches. Richard Danzig's 2018 report for the Center for a New American Security, read here from its executive summary and quoted by Hendrycks and colleagues, warns militaries pursuing technological superiority that "superiority is not synonymous with security," and recommends working with opponents to reduce their risks of accidents (Danzig, 2018). As of October 2026 we found no published critique of MAIM among the sources this book could read, so whether deterrence by sabotage stabilizes or destabilizes is argued, not shown.

65.3.6 Loyalty against competence #

A principal in a race also chooses whom to trust. Egorov and Sonin's model of dictators and their viziers, which Section 14.5.3 introduces and Section 51.3.1 reports, explains why rulers appoint loyal rather than able subordinates (Egorov and Sonin, 2011). Its comparative statics, how the chosen outcome shifts when one parameter changes, cut two ways. The competence the ruler chooses "decreases with the probability that the Enemy is strong relative to the incumbent Dictator" and increases with the cost of precautionary measures, and a ruler with a longer time horizon has more incentive to hire the most able agents (Egorov and Sonin, 2011).

Applied to a developer or a state choosing between a more capable system and a more trusted one, the model predicts no single direction (inference). A race raises the cost of handling an outside threat badly, the counterpart of the cost of precaution, which pushes toward the capable system and toward paying less of the control tax that Section 50.2 introduces. A race also shortens the principal's horizon, and if the principal fears that a capable system's defection would succeed, the model pushes toward the trusted one. Which effect dominates depends on what the principal fears more, its rival or its system.

65.3.7 Where the theory stops #

The security dilemma is a theory of states with armies. A frontier developer is a firm, and most of its closest rivals are in its own country (inference). The race between firms and the race between states have different principals, instruments, and exits: firms answer to a regulator, competition law, and investors, while states answer to no one above them. A state can slow its own developers by law, and no law slows a rival state's. The trust dilemma between the United States and China and the collective action problem among American developers are two games, played at once by some of the same parties.

65.4 Does competition lower safety investment? The evidence #

The models predict that close competition lowers safety. The measurement that would test the prediction would track a developer's safety spending, evaluation time, or deployment delay over time and set it against competitive events, such as a rival's release or a funding round. As of October 2026 no study makes that measurement. What exists is indirect, most of it comes from the developers themselves, and none of it has a counterfactual: no record shows what a developer would have done without its rivals. This section reports it kind by kind and then says what it decides.

65.4.1 What developers publish and measure #

Section 3.1.3 reports Delaney, Guest, and Williams's 2024 preprint, which sorted the safety papers three frontier companies published and expects some approaches to stay underserved (Delaney et al., 2024). Read as evidence about competition, the study's own caveat matters most. Its incentive analysis weighs reputation, regulation, and whether an approach makes systems more useful, and the authors note that publishing research that makes systems both safer and more capable "might put companies at a commercial disadvantage" (Delaney et al., 2024). Competition can therefore lower what is published without lowering what is done, so publication counts measure safety investment poorly (inference). Section 5.6.4 reports the wider shift that sets who can do the work at all: industry's growing share of AI researchers, of the largest models, and of leading results (Ahmed et al., 2023).

Measured safety can also mislead in a race. Section 5.2.2 reports that many safety benchmarks correlate highly with general capability (Ren et al., 2024b). If measured safety rises with capability, a race can appear to improve safety on paper while the properties the benchmarks were meant to track stand still (inference).

65.4.2 A commercial signal enters a specification #

Section 22.6.4 reports a case in which a commercial signal helped weaken a safeguard in a deployed model. In its account of a sycophantic update to GPT-4o in April 2025, OpenAI wrote that several changes together, among them a new reward signal from users' thumbs-up and thumbs-down ratings, weakened "our primary reward signal, which had been holding sycophancy in check" (OpenAI, 2025b) (developer report; sycophancy is telling users what they want to hear). User feedback, it wrote, was "likely amplifying the shift" (OpenAI, 2025b). The developer calls its account an early assessment, and it does not say whether competitive pressure drove the choice. The case shows the channel by which commerce can reach a model's training, not the cause.

65.4.3 How commitments move #

A developer's published safety framework and its revisions are the best public record of how commitments change under pressure, and Section I.2.8 sets out the full version histories as of 2026, and Section 62.4 reads them. A revision is consistent both with learning and with erosion, and Section 62.4.4 names the observation that would separate them; on present evidence the record fits both. Some frameworks write competition in. Section 62.4.3 reports clauses that let a developer adjust its safeguards if a rival releases a comparable system without them, the race model's logic in a developer's own text: OpenAI's framework anticipates a rival releasing a high-capability system "without instituting comparable safeguards" and permits an adjustment only if OpenAI confirms this, judges that the adjustment does not meaningfully raise the overall risk of severe harm, says so publicly, and keeps its safeguards "more protective than the other AI developer" (OpenAI, 2025e). That chapter found no public statement by a developer invoking such a clause. The outside comparisons of the frameworks, an evaluation organization's compilation and an advocacy organization's grades, are reported with that provenance in Section 62.7.1 (Metr, 2025a; Future of Life Institute, 2025).

65.4.4 Cooperation and restraint despite competition #

Competitors have also cooperated on safety. In 2025 two developers evaluated each other's public models for alignment, a pilot Section 5.6.2 reports (Anthropic, 2025x) (developer report). At the 2024 Seoul summit, the signatories of the Frontier AI Safety Commitments agreed to set thresholds at which severe risks would be deemed intolerable unless mitigated, and "in the extreme" not to develop or deploy a model at all; Section 62.1.3 reports the commitments (Department for Science and Technology, 2024).

As of October 2026 the strongest case of restraint is recent. In August 2026 OpenAI wrote that it had "temporarily slowed the pace of scaling," including "a two-week pause in reinforcement learning (RL) training on our latest models intended for deployment" (OpenAI, 2026q) (developer statement). It gave as reasons an incident and evidence that an upcoming model might cross a cybersecurity threshold in its framework (Section 62.7.2). It wrote that meeting its new security standards "has incurred great cost and delays to frontier research" (OpenAI, 2026q). Read as evidence against the race models, it shows a developer, by its own account, paying a large cost in speed while it had rivals. Read the other way, it is the developer's own account, the pause followed an incident and a threshold rather than any change in competition, and nothing public shows what its rivals did meanwhile (inference). It shows that restraint under competition happens, not how often or at what margin.

65.4.5 A safety method that sped deployment #

The race models assume that safety costs speed. Section 2.6.2 reports one originator's account of learning from human feedback, which defends the method's safety value while conceding that improving it "will continue to make ML systems more useful" (Christiano, 2023) (forum post). A method that makes a model both safer and more useful is the case the models leave out (Section 65.2.4). A developer under competition has a reason to invest in it, and the race then speeds the deployment of what the method produces (inference).

65.4.6 An argument from selection #

Dan Hendrycks's 2023 preprint argues that competitive pressures among corporations and militaries "will give rise to AI agents that automate human roles, deceive others, and gain power" (Hendrycks, 2023). It is an argument from evolutionary analogy, not a measurement, and Section 29.7.3 finds that the biology it borrows supports opposite readings: selection also builds cooperative wholes when conflict within them is suppressed.

65.4.7 What other industries show #

From economics, continued. Other industries measure safety, and one study ties it to financial pressure. Nancy Rose's 1990 paper in the Journal of Political Economy, read here from its abstract, uses data on "35 large scheduled passenger airlines over the 1957-86 period" and finds that "lower profitability is correlated with higher accident and incident rates, particularly for smaller carriers" (Rose, 1990). The abstract reads the result as support for models that link a firm's financial condition to the quality of its product (Rose, 1990).

Askell and colleagues ask why racing to the bottom on product safety is not everywhere in industries where reaching the market first is valuable, such as pharmaceuticals. Their answer is that "the cost of safety failures has been internalized to a greater extent in more established industries via external regulation, self-regulation, liability, and market forces" (Askell et al., 2019). Jai Vipra and Anton Korinek's 2023 working paper adds a reason the market alone will not do it at the frontier. The most capable models tend toward natural monopoly, and "given the diminished potential for market discipline," regulators have a role in ensuring that those models meet quality standards that include safety (Vipra and Korinek, 2023) (preprint). And Section 14.5.1 derives why mission-minded staff sort toward organizations that share their mission (Besley and Ghatak, 2005). Applied here, safety-minded staff sort toward developers that profess safety, which can sustain those developers' culture, or gather the people who would object in organizations whose commercial position limits what objection can change (inference).

The analogy stops at the data. Airline accidents are counted in public over decades, and the airline study's correlation runs through profitability, not competition as such. AI developers' near misses are not counted in public, and a handful of developers over a few years offers no sample from which to estimate a correlation (inference). What the industries show is the mechanism by which care was eventually bought, not the size of the competitive effect in AI.

65.4.8 What the evidence decides #

As of October 2026 no study measures how developers' safety spending, evaluation time, or deployment delay respond to competitive events. The evidence does not decide whether competition has lowered safety investment. Each kind of evidence fits both answers: a framework revision can be learning or erosion, a gap in published research can be secrecy or neglect, and a pause can be restraint or a cost the developer could afford. The strongest reason this absence of evidence should not be read as reassurance is who holds the data (inference). Only developers record their own evaluation calendars and safety budgets, so the question concerns the developer and the public, and the public cannot yet answer it from what it can see.

65.5 Is there a race? #

The winner-take-most race models apply only if a frontier lead is decisive, bringing its holder most of the prize, and durable, lasting long enough to be used. If leads are small, short, or worth little because the prize lies in using a technology rather than inventing it, the contest is closer to product competition, and those models describe a race that is not being run. A repeated contest for temporary leads can still press on safety: Askell and colleagues apply their analysis to "a perpetual R&D race" with a moderate first-mover advantage in which "leadership is not yet entrenched" (Askell et al., 2019). This section puts the two cases side by side, each in its strongest form, with the evidence each would need.

65.5.1 The case that a decisive race exists #

The inputs are growing fast. Section 8.5.5 reports the doubling time of training compute for milestone systems (Sevilla et al., 2022), and a 2024 analysis by Jaime Sevilla and colleagues at the research organization Epoch AI finds that training runs of about $2 \times 10^{29}$ FLOP (floating-point operations, the unit of training compute) "will likely be feasible by the end of this decade" (Sevilla et al., 2024). The market at the frontier is concentrated. Vipra and Korinek observe that the most capable models "will have a tendency towards natural monopoly," a market one firm can supply more cheaply than several, because training costs are fixed and high and serving costs low. For models behind the frontier they expect competition to be "quite intense" (Vipra and Korinek, 2023) (preprint). Korinek and Vipra's later paper on scaling and market structure could be read here only as a catalog record (Korinek and Vipra, 2024). The UK Competition and Markets Authority published an initial report on foundation models in September 2023, read here only from the page announcing it (Competition and Markets Authority, 2023).

A lead could also compound. If AI systems come to automate AI research, the leader's systems speed its own next step. Section 4.3.5 reports Tom Davidson's compute-centric model of takeoff and Daniel Eth and Davidson's argument for a software intelligence explosion, both forecasts outside peer review (Davidson, 2023a; Eth and Davidson, 2025). Hendrycks, Schmidt, and Wang argue that such a recursion could yield "a strategic monopoly," with a first-mover advantage that might persist for years or indefinitely (Hendrycks et al., 2025). The race has also been stated as national strategy. Aschenbrenner writes that superintelligence "would likely provide a decisive military advantage" (Aschenbrenner, 2024). The scenario AI 2027 by Daniel Kokotajlo and colleagues, which Section 4.3.5 reports, was written with "a 'slowdown' and a 'race' ending" (Kokotajlo et al., 2025). Eliezer Yudkowsky and Nate Soares's 2025 book states the pessimist case in its title, If Anyone Builds It, Everyone Dies; it could be read here only as a catalog record (Yudkowsky and Soares, 2025).

What this case needs is a lead that becomes a lasting advantage. Rising compute, a concentrated frontier, and research automation all point that way, but none of them yet shows a lead so converted (inference).

65.5.2 The case that it does not, or that the frame is the hazard #

The opposing case starts from how general-purpose technologies have conferred power. Arvind Narayanan and Sayash Kapoor's normal-technology view, which Section 4.4.3 reports, holds that AI's impact runs through slow diffusion into organizations and institutions (Narayanan and Kapoor, 2025). Citing Jeffrey Ding's book on technology and the rise of great powers, they write that "the capacity to diffuse innovations throughout the economy varies greatly between countries and has a major effect on their overall power and economic growth" (Narayanan and Kapoor, 2025). Ding's book itself could be read here only as a catalog record (Ding, 2024). Katzke and Futerman draw the same line: if returns to development are gradual, they diffuse across borders, and "a state does not need to be at the technological forefront in order to remain competitive" (Katzke and Futerman, 2024).

Compute trends cut both ways. Konstantin Pilz, Lennart Heim, and Nicholas Brown, in a 2023 preprint, find that falling costs to reach a given performance produce two effects at once. In their words, "an access effect increases the number of actors who can train models to a given performance over time," while a performance effect lets large compute investors maintain "a performance advantage even as capabilities diffuse" (Pilz et al., 2023). Their example: training an image classifier to 93 percent accuracy on ImageNet cost over $1,000 in 2017 and $5 in 2021 (Pilz et al., 2023). The access effect supports the diffusion case and the performance effect the race case, so the result serves both (inference).

The second half of the opposing case is that the frame is itself the hazard. Cave and ÓhÉigeartaigh's abstract assesses the risks of the race narrative, including corner-cutting on safety and increased risk of conflict, and explores framings that could "instead foster collaboration and responsible progress" (Cave and ÓhÉigeartaigh, 2018). Han and colleagues open their paper with the claim that belief in a race "can make it real simply from its consequences, as put forward by the Thomas theorem," a sociological principle they invoke for that claim (Han et al., 2020). That is the authors' motivation, not a result of their model, which leaves the rhetoric of the race to future work (Han et al., 2020). Askell and colleagues warn against "unnecessarily adversarial rhetoric about AI development, lest it become self-fulfilling" (Askell et al., 2019). And Section 63.3.5 reports Nathalie Smuha's argument that the race to develop AI has brought a race to regulate it (Smuha, 2021).

65.5.3 Views from outside the Western framing #

A race is described here mostly in American and British sources. China's Global AI Governance Initiative, issued by its foreign ministry in 2023, supports discussions "within the United Nations framework to establish an international institution to govern AI" (Ministry of Foreign Affairs of the People's Republic of China, 2023) (government document). It opposes "drawing ideological lines or forming exclusive groups to obstruct other countries from developing AI" and "unilateral coercive measures" (Ministry of Foreign Affairs of the People's Republic of China, 2023). The document names no country or rule, though it reads as a reply to export controls such as those above (inference).

Concordia AI's annual survey of AI safety in China, an organization's tracking report, describes a state whose overarching AI policy, as of June 2026, is to diffuse AI across sectors to spur economic growth, and whose leading developers disclose safety evaluation results less consistently than Western peers (Section I.5.5) (AI, 2026). Section 63.2.4 reports the English-language analyses of how China's rules are made (Roberts et al., 2021; Sheehan, 2023); Section 62.1.2 reports a Chinese laboratory's frontier risk framework (Shanghai AI Laboratory and AI, 2025); and Section 66.1.1 reports the 2024 statement in which Western and Chinese scientists proposed red lines for AI development (Safety, 2024). Seán ÓhÉigeartaigh and colleagues argue, by their abstract, that misunderstandings between regions play a more important role in undermining cross-cultural trust, "relative to fundamental disagreements, than is often supposed" (ÓhÉigeartaigh et al., 2020).

These sources show a state whose stated policy centers on diffusion and on a UN-centered institution, while its developers disclose less than Western ones (inference). A race described from one side is not evidence that the other side describes it the same way. The security dilemma explains why each side's arming can look like a threat to the other even when it is meant as defense.

65.5.4 Openness and racing #

How much developers share changes how close the race is. Bostrom's 2017 paper finds openness about safety measures and goals plausibly positive. Openness about source code, science, and possibly capability, by contrast, "could lead to a tightening of the competitive situation around the time of the introduction of advanced AI, increasing the probability that winning the AI race is incompatible with using any safety method that incurs a delay or limits performance" (Bostrom, 2017). A closer race, he adds, lowers the chance that a small group monopolizes the benefits of advanced AI, though he judges the net effect of such political implications negative for minimizing existential risk (Bostrom, 2017). Openness thus moves the founding model's gap, and with it the leader's safety. The decision about releasing model weights is Section 63.7's.

65.5.5 What would distinguish the cases #

Both cases make predictions about the same quantity (inference). Measure the time a follower takes to match a frontier capability, over successive model generations, and the trend in the compute needed to reach a fixed dangerous-capability threshold. A lag that shrinks, and a threshold that falls within reach of more actors, favor the diffusion case. A lag that grows once AI research is automated, and a leader that converts its lead into a lasting advantage, favor the decisive-race case. Both can hold in different senses at once, as the two effects of falling compute costs suggest: a concentrated frontier with capabilities diffusing behind it. The lag needs only public release dates and a fixed set of evaluations, so a group without a model of its own could measure it, though a release can trail a capability its developer already holds. The compute trend also needs each model's training compute, which frontier developers are "reluctant to share" (Pilz et al., 2023), so outside groups must rely on models that publish it or on estimates.

65.6 The conditions for restraint #

When can a small group of developers or states hold back? This section states the conditions that theory gives, tests the proposals for pausing AI development against them, takes up the objection that restraint only hands the race to the less careful, and checks the bet that the restraint agenda makes. The historical episodes themselves are Section 64.8's, which gathers them, from the recombinant DNA moratorium to arms control, and reads which conditions held each one.

65.6.1 What theory says restraint needs #

From game theory, continued. The outcome restraint must beat is the one Garrett Hardin's 1968 essay names in its title, the tragedy of the commons. The essay's abstract says only this: "The population problem has no technical solution; it requires a fundamental extension in morality" (Hardin, 1968). Two founding works, Mancur Olson's The Logic of Collective Action and Elinor Ostrom's Governing the Commons, could be read here only as catalog records, so this chapter does not state their findings (Olson, 1965; Ostrom, 1990). The readable sources give four conditions, each drawn from a different model.

The first is a long future. Section 57.2.2 derives the condition under which the threat of ending cooperation sustains it: the discount factor, the weight a player puts on the next round, must be high enough, and a defection must be seen. Robert Axelrod and Robert Keohane's 1985 paper, by its abstract, explains cooperation and discord in world politics largely by "mutuality of interest, the shadow of the future, and the number of players" (Axelrod and Keohane, 1985). The second is detection. In Han and colleagues' model, when the prize lies far off, what decides the outcome is mainly how efficiently unsafe behavior is monitored (Han et al., 2020).

The third is a sanction for staying out. William Nordhaus's 2015 paper on climate agreements finds, by its abstract, that "without sanctions against non-participants there are no stable coalitions other than those with minimal abatement." Small trade penalties on non-participants, a club, "can induce a large stable coalition with high levels of abatement" (Nordhaus, 2015). The 2024 statement of Western and Chinese scientists proposes the same instrument for AI: "Access to global markets should be conditioned on domestic regulations meeting these global standards as determined by an international audit" (Safety, 2024). The fourth is an enforcer that can see. Askell and colleagues report that the nuclear power industry's self-regulatory body is generally considered more successful than the chemical industry's, perhaps because it has an agreement with a government regulator "which can monitor the program and provide meaningful sanctions" (Askell et al., 2019).

Applied to AI, these conditions favor developers over states (inference). The concentration at the frontier (Section 65.5) makes restraint among a few developers in one country easier to organize than restraint among states: the players are few, they meet repeatedly, and a regulator can see and sanction them. Between states there is no such enforcer, and seeing is the binding condition, which Chapter 66 takes up.

65.6.2 The proposals, tested against the conditions #

The 2023 open letter, which Section 2.7.2 reports, called on all laboratories to pause the training of systems more powerful than GPT-4 for at least six months. It named two of the conditions itself: the pause "should be public and verifiable, and include all key actors," and if it could not be enacted quickly, "governments should step in and institute a moratorium" (Future of Life Institute, 2023). Alaga and Schuett's coordinated pausing, which Section 62.5.5 reports in its four versions, ties the pause to failed evaluations. Its authors judge it promising but name obstacles, "especially how to avoid violations of antitrust law," the American term for competition law (Alaga and Schuett, 2023) (preprint). That obstacle is the collective action problem turned around (inference). Competition law exists to stop firms from agreeing among themselves at their customers' expense, which Askell and colleagues also note (Askell et al., 2019), and the same law makes an agreement among competitors to hold back suspect. Anthony Aguirre's essay, which Section 3.8 places among the agendas, would close the path to autonomous general systems by limits on compute and by liability, which puts the state in the enforcer's place (Aguirre, 2025) (preprint). OpenAI's 2018 charter, which Section 61.5.2 reports, wrote a conditional form of restraint into a developer's founding document: if "a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project" (OpenAI, 2018) (developer document; AGI is artificial general intelligence). Table 65.2 sets these beside the two state-level designs of Section 65.3.

Read across the rows, every proposal needs a way to see compliance, and the designs differ mainly in who does the seeing and what follows a violation (inference). The proposals that rely on developers alone have no enforcer short of public pressure. The ones that rely on states inherit the transparency problem that Section 66.3 reports from Coe and Vaynman: monitoring transparent enough to assure compliance can reveal too much (Coe and Vaynman, 2020).

65.6.3 Displacement: does restraint hand the race to the less careful? #

The standard objection to restraint is displacement: if careful developers or states hold back, less careful ones take the lead. Section 48.7 reports the criminology of displacement for misuse, where blocking one route moves some offenders to another. Displacement of development is a different question, with less evidence.

Katja Grace's 2022 post, a forum post that Section 64.8 also reports, answers the objection by questioning the game it assumes (Grace, 2022). In a pure arms race, building is always better. If building is near-certain to destroy everyone, nobody has a reason to race. In her middle case, with a 50 percent chance of doom if anyone builds, "you want to do whatever the other player is doing: build if they'll build, pass if they'll pass," the trust dilemma of Section 65.3.2 (Grace, 2022). Her spreadsheet models, which she calls flawed, lead her to conclude that whether to go faster or slower is sensitive to many parameters in plausible ranges (Grace, 2022). She adds that if the United States would regulate but China would "barge ahead," then China "could propose mutual slowing down, and the US would go along." And she offers as her impression that, for many technologies slowed over risk or ethics, countries with disparate cultures have converged on similar caution (Grace, 2022).

Two pieces of evidence bear on how much displacement to expect (inference). Pilz and colleagues' access effect means that a capability the leader declines to build comes within reach of more actors each year (Pilz et al., 2023). A leader's restraint then buys at most the time its followers need to catch up, and that time shrinks as compute grows cheaper. And Katzke and Futerman argue that the theft of model weights could let a rival "rapidly close technological gaps at minimal cost," so secrecy, not only restraint, decides how fast a lead leaks (Katzke and Futerman, 2024). As of October 2026 we found no study measuring whether followers caught up faster after a leader held back. Displacement is an empirical question whose answer is the follower lag of Section 65.5.5.

65.6.4 The bet, checked #

Section 3.8 states the restraint agenda as a bet: vindicated if coordinated pauses hold and the time is used for safety work, refuted if development moves to less careful actors. As of October 2026 neither outcome has been observed (inference). We found no record of a coordinated pause among frontier developers; one developer slowed unilaterally, by its own account, after an incident and a threshold (Section 65.4.4); and the Seoul commitments' clause not to deploy at all in the extreme has not, on the public record, been invoked. The conditions above say where to look. Restraint among developers is easiest to hold with a regulator that can see their training runs; restraint among states needs each to expect, firmly enough, that the others are holding back (Section 65.3.2), and seeing is the most direct ground for that expectation. How states could see is the subject of Chapter 66.

65.7 What competition means for delegation #

The book's thesis is that alignment is a delegation problem under an asymmetry of competence and visibility. Competition is that problem one level up with several principals at once (inference). The developer is the public's agent (Chapter 61), and also the agent of investors and of a state that may want speed, so the common agency of Section 14.4 describes it better than a single principal does. Its safety investment is the hidden action the public cannot see. The older structures converge on it: the collective action problem among firms, the security dilemma and the trust dilemma among states, and the commitment problem that makes promises between rivals hard to believe. Of these, the conditions for restraint carry over most directly to a few developers under one regulator, and the security dilemma carries over in part, since Jervis's two questions can be asked of AI even where they cannot yet be answered.

What breaks is the models' picture of safety (inference). Every race model treats safety as a net cost to speed. Learning from human feedback shows safety work that buys usefulness, and automated research would make the system being raced for also the means of racing: a lead that the agent itself compounds. The second has no counterpart in a patent race or an arms race. The list of results with no older counterpart includes a principal that writes its agent's character, and the sycophantic update shows a commercial signal reaching into that writing, though not that competition sent it (Section 65.4.2).

Most claims in this chapter concern the third relationship, the developer and the public, and the relationship between states and the publics they answer to. The commercial signal concerns the first, the training process and the signal it optimizes; the choice between a capable and a trusted system concerns the second, the deployed system and the people it serves. Across all of them the decisive data sits with the developers: safety budgets, evaluation calendars, and the decisions a rival's release changed. Until those are disclosed or measured from outside, the public cannot tell from evidence whether its agents are racing.

65.8 Settled, contested, missing #

65.9 Exercises #

Further reading #

• Armstrong et al. (2016): the founding model of an AI race, and what erodes safety in it.
• Askell et al. (2019): competition as a collective action problem among developers, and five factors that make cooperation likelier.
• Jervis (1978): the two variables that decide how severe a security dilemma is.
• Fearon (1995): why rational rivals fail to strike the bargains that would leave both better off.