AI Alignment
Part XI: Synthesis

Synthesis

In brief

Ideas
  • Does the thesis hold? It keeps its problem claim, with 41 of 69 older diagnoses carrying over, and gives up the strong form of its convergence claim, with 9 of 69 remedies carrying or reversed (Ch. 69).
  • The technical program rebuilds older remedies in form; remedies broke most often where an agent adapts to its check or a developer holds roles older fields kept apart (§69.7; §69.3.2).
  • Fifty-nine chapter cruxes merge into twenty-three debates that collapse onto a few designs: one long training run, planted against natural failures, unrecognizable observation, and a stronger adversary against a weaker judge (Ch. 70).
  • Ranked by the cruxes an answer would decide or move, three open problems lead: checking developers from records, planted against natural failures, and long reinforcement learning (Ch. 71).
Evidence
  • Settled: what meets the book's bar shows failures can occur or instruments are imperfect; none of it gives a deployment rate or a method that holds as the capability gap grows (§70.2).
  • Reported: every verdict on what carries over is the book's classification of its chapters' findings; no study was designed to test whether a borrowed structure transfers (§69.8).
Open
  • Crux: does the persona account survive large-scale reinforcement learning? The thesis's verdict on the deployed system rides on it (§70.7).
  • Open: an agent that outgrows its principal's ability to check it, on which no borrowed field has evidence; the top open problems wait on developers' access (§69.7; §71.6).

This part reads the rest of the book as evidence about three questions. The first is whether the thesis holds: every chapter that borrowed from another field said where the analogy stopped, and Chapter 69 collects those verdicts, tests the delegation thesis assumption by assumption for each of the three relationships, and gives its verdict on the results the book listed before drafting as having no counterpart in the older fields. The second is what the evidence has and has not settled: Chapter 70 merges the chapters' cruxes into the debates they belong to, sets out each with its positions and the result that would decide it, and lists what the book counts as settled, under a rule of independent replication and the refinements it states for theorems, existence results, and meta-analyses. The third is what to do next: Chapter 71 ranks the open problems by a stated criterion, the number of those cruxes an answer would decide or move, and names who could work on each.

The part comes last because it introduces no new evidence; it recalls each result by cross-reference and judges the whole. It serves all four problems at once, since its subject is the loop between them: the couplings by which verification shapes instillation, verifiability shapes specification, and maintenance limits what can be verified, with competition shaping specification on the book's reading of one developer's account. Its conclusions are written so that they could have come out otherwise, and where the evidence does not decide, the part says so rather than leaning.

It assumes the rest of the book, and in particular the thesis and its negative space (Section 1.7), the rules of evidence and transfer of Chapter 5, and the delegation models and their assumptions in Chapter 14.

Chapters in this part

  1. 69 What Carries Over and What Breaks

    The book's thesis, that alignment is delegation under an asymmetry of competence and visibility, tested assumption by assumption and analogy by analogy against the drafted chapters. Older diagnoses carried far more often than older remedies; of the five results fixed in advance as having no counterpart in older fields, one split, one narrowed, and three stood; and the two most frequent reasons a borrowed remedy broke, an agent that adapts to its measurement and a developer that holds roles older fields kept apart, are not on the thesis's list of what is new.

  2. 70 Cruxes

    The debates the book's evidence has not settled, collected from every chapter's crux and merged into twenty-three where chapters ask the same question: for each, the positions, the result that would decide it, and its status as of October 2026, with the evidence left to the chapters that report it. Also how short the list of settled results is and what kind of results fill it, the experiments that would move several debates at once, and the results that would show the book's conclusions wrong.

  3. 71 Open Problems

    The questions the book's evidence leaves open, ranked by a stated criterion: how many of the chapters' cruxes an answer would decide or move, counted from the designs of the cruxes chapter. Twenty problems, led within two points of one another by whether developers' commitments, claims, and restraint can be checked from records, whether methods validated on planted failures work on failures that arise on their own, and what long reinforcement learning does to a model's dispositions. For each, who could work on it: machine learning research, another discipline, or an institution. Then the gaps the chapters marked missing, sorted by who could supply them; the problems that need fields outside machine learning; the research agendas' bets revisited against the evidence; and the problems whose bottleneck is access rather than method.

References for Part XI

193 works cited across this part's chapters.

  1. Abiri, G. (2024). Public Constitutional AI. Elsevier BV. preprint Ch. 70
  2. AI Security Institute (2026c). Cheating behaviour in frontier model evaluations. AI Security Institute. non-peer-reviewed Ch. 70
  3. Akerlof, G. A., and Kranton, R. E. (2005). Identity and the Economics of Organizations. Journal of Economic Perspectives. Ch. 69 Ch. 71
  4. Amodei, D. (2025). The Urgency of Interpretability. darioamodei.com. non-peer-reviewed Ch. 70
  5. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., and Mané, D. (2016). Concrete Problems in AI Safety. arXiv. preprint Ch. 71
  6. Ankeny, R. A., and Leonelli, S. (2011). What’s so special about model organisms? Studies in History and Philosophy of Science Part A. Ch. 69
  7. Anthropic (2023e). Interpretability Dreams. Transformer Circuits Thread. non-peer-reviewed Ch. 70
  8. Anthropic (2023f). The Long-Term Benefit Trust. Anthropic. non-peer-reviewed Ch. 70
  9. Anthropic (2024i). Claude’s Character. Anthropic. non-peer-reviewed Ch. 69
  10. Anthropic (2025at). Why do we take LLMs seriously as a potential source of biorisk? Anthropic. non-peer-reviewed Ch. 70
  11. Anthropic (2026at). Teaching Claude why. Anthropic. non-peer-reviewed Ch. 69 Ch. 70
  12. Anthropic (2026au). The persona selection model. Anthropic. non-peer-reviewed Ch. 70
  13. Anthropic (2026c). AI models may be capable of self-replicating without access to a datacenter. Anthropic. non-peer-reviewed Ch. 70
  14. Anthropic (2026r). Claude's Constitution. Anthropic. non-peer-reviewed Ch. 70
  15. Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., … Krueger, D. (2024). Foundational Challenges in Assuring Alignment and Safety of Large Language Models. arXiv. preprint Ch. 71
  16. Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., and Nanda, N. (2024). Refusal in Language Models Is Mediated by a Single Direction. Advances in Neural Information Processing Systems. Ch. 69
  17. Armstrong, S., Bostrom, N., and Shulman, C. (2016). Racing to the precipice: a model of artificial intelligence development. AI & SOCIETY. Ch. 70
  18. Arnesen, S., Rein, D., and Michael, J. (2024). Training Language Models to Win Debates with Self-Play Improves Judge Accuracy. arXiv. preprint Ch. 70
  19. Bainbridge, L. (1983). Ironies of automation. Automatica. Ch. 70
  20. Baker, B., Huizinga, J., Gao, L., Dou, Z., Guan, M. Y., Madry, A., … Farhi, D. (2025). Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation. arXiv. preprint Ch. 69 Ch. 70
  21. Balesni, M., Korbak, T., and Evans, O. (2024a). Lessons from Studying Two-Hop Latent Reasoning. arXiv. preprint Ch. 70
  22. Barnes, B. (2020). Debate update: Obfuscated arguments problem. AI Alignment Forum. non-peer-reviewed Ch. 70
  23. Bartha, P. (2010). By Parallel Reasoning. Oxford University Press. Ch. 69
  24. Bebchuk, L. A., and Tallarita, R. (2020). The Illusory Promise of Stakeholder Governance. Elsevier BV. preprint Ch. 70
  25. Belrose, N., and Pope, Q. (2024). Counting arguments provide no evidence for AI doom. AI Alignment Forum. non-peer-reviewed Ch. 70
  26. Bem, D. J. (1972). Self-Perception Theory. Advances in Experimental Social Psychology. Ch. 69
  27. Bengio, Y., Maharaj, T., Ong, L., Russell, S., Song, D., Tegmark, M., … Žikelić, D. (2025e). The Singapore Consensus on Global AI Safety Research Priorities. arXiv. preprint Ch. 71
  28. Betley, J., Warncke, N., Sztyber-Betley, A., Tan, D., Bao, X., Soto, M., … Evans, O. (2026a). Training large language models on narrow tasks can lead to broad misalignment. Nature. Ch. 69
  29. Bhatt, A., Rushing, C., Kaufman, A., Tracy, T., Georgiev, V., Matolcsi, D., Khan, A., and Shlegeris, B. (2025). Ctrl-Z: Controlling AI Agents via Resampling. arXiv. preprint Ch. 69
  30. Birch, J. (2024). The Edge of Sentience. Oxford University PressOxford. Ch. 70
  31. Black, S., Stickland, A. C., Pencharz, J., Sourbut, O., Schmatz, M., Bailey, J., … Cooney, A. (2025). RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agents. arXiv. preprint Ch. 70
  32. Bond, C. F., and DePaulo, B. M. (2006). Accuracy of Deception Judgments. Personality and Social Psychology Review. Ch. 69
  33. Bondarenko, A., Volk, D., Volkov, D., and Ladish, J. (2025). Demonstrating specification gaming in reasoning models. arXiv. preprint Ch. 70
  34. Bowkis, A., Buhl, M. D., Pfau, J., and Irving, G. (2026). Automated alignment is harder than you think. arXiv. preprint Ch. 70
  35. Bowman, S. R. (2025). Putting up Bumpers. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 71
  36. Brown-Cohen, J., Irving, G., and Piliouras, G. (2024). Scalable AI Safety via Doubly-Efficient Debate. International Conference on Machine Learning. Ch. 69
  37. Brysse, K., Oreskes, N., O’Reilly, J., and Oppenheimer, M. (2013). Climate change prediction: Erring on the side of least drama? Global Environmental Change. Ch. 70
  38. Bucknall, B., Trager, R. F., and Osborne, M. A. (2025). Position: Ensuring mutual privacy is necessary for effective external evaluation of proprietary AI systems. arXiv. preprint Ch. 71
  39. Budescu, D. V., Broomell, S., and Por, H.-H. (2009). Improving Communication of Uncertainty in the Reports of the Intergovernmental Panel on Climate Change. Psychological Science. Ch. 70
  40. Burns, C., Izmailov, P., Kirchner, J. H., Baker, B., Gao, L., Aschenbrenner, L., … Wu, J. (2024). Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision. International Conference on Machine Learning. Ch. 70
  41. Butlin, P., Long, R., Elmoznino, E., Bengio, Y., Birch, J., Constant, A., … VanRullen, R. (2023). Consciousness in Artificial Intelligence: Insights from the Science of Consciousness. arXiv. preprint Ch. 70
  42. Calvano, E., Calzolari, G., Denicolò, V., and Pastorello, S. (2020). Artificial Intelligence, Algorithmic Pricing, and Collusion. American Economic Review. Ch. 70
  43. Cameron, J., and Pierce, W. D. (1994). Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis. Review of Educational Research. Ch. 69
  44. Canavan, C., Shrivastava, A., Qi, A., Michala, J., and Roger, F. (2026). Three Concrete Challenges and Two Hopes for the Safety of Unsupervised Elicitation. arXiv. preprint Ch. 70
  45. Carey, R. (2018). Incorrigibility in the CIRL Framework. Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. Ch. 70
  46. Carlsmith, J. (2023). Scheming AIs: Will AIs fake alignment during training in order to get power? arXiv. preprint Ch. 70
  47. Carroll, M., Dragan, A., Russell, S., and Hadfield-Menell, D. (2022). Estimating and Penalizing Induced Preference Shifts in Recommender Systems. International Conference on Machine Learning. Ch. 69
  48. Carroll, M., Foote, D., Siththaranjan, A., Russell, S., and Dragan, A. (2024). AI Alignment with Changing and Influenceable Reward Functions. International Conference on Machine Learning. Ch. 70
  49. Cartwright, N., and Hardie, J. (2012). Evidence-Based Policy. Oxford University Press. Ch. 69
  50. Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., … Hadfield-Menell, D. (2023). Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback. arXiv. preprint Ch. 71
  51. Casper, S., Ezell, C., Siegmann, C., Kolt, N., Curtis, T. L., Bucknall, B., … Hadfield-Menell, D. (2024a). Black-Box Access is Insufficient for Rigorous AI Audits. The 2024 ACM Conference on Fairness, Accountability, and Transparency. Ch. 71
  52. Cave, S., and ÓhÉigeartaigh, S. S. (2018). An AI Race for Strategic Advantage. Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. Ch. 70
  53. Chandler, J., Mueller, P., and Paolacci, G. (2014). Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods. Ch. 69
  54. Chanin, D., Wilken-Smith, J., Dulka, T., Bhatnagar, H., Golechha, S., and Bloom, J. (2024). A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders. Neural Information Processing Systems. Ch. 69
  55. Chaudhary, M., Su, I., Hooda, N., Shankar, N., Tan, J., Zhu, K., … Panda, A. (2025). Evaluation Awareness Scales Predictably in Open-Weights Large Language Models. arXiv. preprint Ch. 70
  56. Christiano, P. (2022b). Where I agree and disagree with Eliezer. AI Alignment Forum. non-peer-reviewed Ch. 70
  57. Cloud, A., Le, M., Chua, J., Betley, J., Sztyber-Betley, A., Hilton, J., Marks, S., and Evans, O. (2025). Subliminal Learning: Language models transmit behavioral traits via hidden signals in data. arXiv. preprint Ch. 69
  58. Costello, T. H., Pennycook, G., and Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. Science. Ch. 69
  59. Cotra, A. (2022). Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover. AI Alignment Forum. non-peer-reviewed Ch. 70
  60. Critch, A., and Krueger, D. (2020). AI Research Considerations for Human Existential Safety (ARCHES). arXiv. preprint Ch. 69 Ch. 70 Ch. 71
  61. Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., and Graepel, T. (2020). Open Problems in Cooperative AI. arXiv. preprint Ch. 71
  62. Davidson, T., Finnveden, L., and Hadshar, R. (2025). AI-Enabled Coups: How a Small Group Could Use AI to Seize Power. Forethought. non-peer-reviewed Ch. 69
  63. Debenedetti, E., Shumailov, I., Fan, T., Hayes, J., Carlini, N., Fabian, D., … Tramèr, F. (2026). Defeating Prompt Injections by Design. 2026 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). Ch. 70
  64. Deci, E. L., Koestner, R., and Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin. Ch. 69
  65. Dixit, A., Grossman, G. M., and Helpman, E. (1997). Common Agency and Coordination: General Theory and Application to Government Policy Making. Journal of Political Economy. Ch. 69
  66. Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., and Gardner, M. (2021). Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus. arXiv. preprint Ch. 69
  67. Dominguez-Olmedo, R., Hardt, M., and Mendler-Dünner, C. (2024). Questioning the Survey Responses of Large Language Models. Advances in Neural Information Processing Systems. Ch. 70
  68. Duflo, E., Greenstone, M., Pande, R., and Ryan, N. (2013). Truth-telling by Third-party Auditors and the Response of Polluting Firms: Experimental Evidence from India*. The Quarterly Journal of Economics. Ch. 69 Ch. 71
  69. Eckhardt, D., and Lee, L. (1985). A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors. IEEE Transactions on Software Engineering. Ch. 69
  70. Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., … Olah, C. (2022b). Toy Models of Superposition. Transformer Circuits Thread. non-peer-reviewed Ch. 69
  71. Emmons, S., Jenner, E., Elson, D. K., Saurous, R. A., Rajamanoharan, S., Chen, H., Shafkat, I., and Shah, R. (2025). When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors. arXiv. preprint Ch. 70
  72. Engels, J., Baek, D., Kantamneni, S., and Tegmark, M. (2025). Scaling Laws For Scalable Oversight. Advances in Neural Information Processing Systems 38. doi:10.52202/085713-3254. Ch. 70 Ch. 71
  73. Farrell, H., Gopnik, A., Shalizi, C., and Evans, J. (2025). Large AI models are cultural and social technologies. Science. Ch. 70
  74. Fearon, J. D. (1999). Electoral Accountability and the Control of Politicians: Selecting Good Types versus Sanctioning Poor Performance. Democracy, Accountability, and Representation. Ch. 69
  75. Gardner-Challis, N., Bostock, J., Kozhevnikov, G., Sinclaire, M., Velja, J., Abate, A., and Griffin, C. (2026). When can we trust untrusted monitoring? A safety case sketch across collusion strategies. arXiv. preprint Ch. 70
  76. Garfinkel, B., and Dafoe, A. (2019). How does the offense-defense balance scale? Journal of Strategic Studies. Ch. 70
  77. Gebru, T., and Torres, É. P. (2024). The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence. First Monday. Ch. 70
  78. Goldwasser, S., Kim, M. P., Vaikuntanathan, V., and Zamir, O. (2022). Planting Undetectable Backdoors in Machine Learning Models : [Extended Abstract]. 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS). Ch. 69
  79. Grace, K., Stewart, H., Sandkühler, J. F., Thomas, S., Weinstein-Raun, B., Brauner, J., and Korzekwa, R. C. (2024). Thousands of AI Authors on the Future of AI. arXiv. preprint Ch. 70
  80. Greenblatt, R., and Shlegeris, B. (2024). The case for ensuring that powerful AIs are controlled. LessWrong / AI Alignment Forum. non-peer-reviewed Ch. 70
  81. Greenblatt, R., Roger, F., Krasheninnikov, D., and Krueger, D. (2024c). Stress-Testing Capability Elicitation With Password-Locked Models. Advances in Neural Information Processing Systems. Ch. 70
  82. Gururangan, S., Card, D., Dreier, S., Gade, E., Wang, L., Wang, Z., Zettlemoyer, L., and Smith, N. A. (2022). Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Ch. 69
  83. Guttman, L. (1955). The Determinacy of Factor Score Matrices with Implications for Five Other Basic Problems of Common‐factor Theory 1. British Journal of Statistical Psychology. Ch. 69
  84. Hackenburg, K., Tappin, B. M., Hewitt, L., Saunders, E., Black, S., Lin, H., … Summerfield, C. (2025). The levers of political persuasion with conversational artificial intelligence. Science. Ch. 69
  85. Hadfield-Menell, D., and Hadfield, G. K. (2019). Incomplete Contracting and AI Alignment. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. Ch. 69 Ch. 71
  86. Hadfield-Menell, D., Dragan, A., Abbeel, P., and Russell, S. (2017b). The Off-Switch Game. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. Ch. 70
  87. Hägele, A., Gema, A. P., Sleight, H., Perez, E., and Sohl-Dickstein, J. (2026). The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity? arXiv. preprint Ch. 69 Ch. 70
  88. Hammond, L., Chan, A., Clifton, J., Hoelscher-Obermaier, J., Khan, A., McLean, E., … Rahwan, I. (2025). Multi-Agent Risks from Advanced AI. arXiv. preprint Ch. 70
  89. Hebbar, V. (2025b). How training-gamers might function (and win). AI Alignment Forum. non-peer-reviewed Ch. 70
  90. Hebbar, V., and Hubinger, E. (2022). Path dependence in ML inductive biases. AI Alignment Forum. non-peer-reviewed Ch. 70
  91. Hendrycks, D., and Hiscott, L. (2025). The Misguided Quest for Mechanistic AI Interpretability. AI Frontiers. non-peer-reviewed Ch. 70
  92. Hendrycks, D., Carlini, N., Schulman, J., and Steinhardt, J. (2021c). Unsolved Problems in ML Safety. arXiv. preprint Ch. 71
  93. Hendrycks, D., Schmidt, E., and Wang, A. (2025). Superintelligence Strategy: Expert Version. arXiv. preprint Ch. 70
  94. Hesse, M. B. (1963). Models and Analogies in Science. Sheed & Ward. Ch. 69
  95. Himmelreich, J. (2023). Against “Democratizing AI”. AI & SOCIETY. Ch. 70
  96. Holmstrom, B., and Milgrom, P. (1991). Multitask Principal–Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design. The Journal of Law, Economics, and Organization. Ch. 69
  97. Howard, R. (1966). Information Value Theory. IEEE Transactions on Systems Science and Cybernetics. Ch. 71
  98. Huang, S., Siddarth, D., Lovitt, L., Liao, T. I., Durmus, E., Tamkin, A., and Ganguli, D. (2024a). Collective Constitutional AI: Aligning a Language Model with Public Input. The 2024 ACM Conference on Fairness, Accountability, and Transparency. Ch. 70
  99. Hubinger, E. (2022). How likely is deceptive alignment? AI Alignment Forum. non-peer-reviewed Ch. 70
  100. Hubinger, E. (2025). Alignment remains a hard, unsolved problem. AI Alignment Forum. non-peer-reviewed Ch. 71
  101. Hughes, J., Price, S., Lynch, A., Schaeffer, R., Barez, F., Koyejo, S., … Sharma, M. (2024). Best-of-N Jailbreaking. arXiv. preprint Ch. 70
  102. Humlum, A., and Vestergaard, E. (2025). Large Language Models, Small Labor Market Effects. Elsevier BV. preprint Ch. 70
  103. Irpan, A., Turner, A. M., Kurzeja, M., Elson, D. K., and Shah, R. (2025). Consistency Training Helps Stop Sycophancy and Jailbreaks. arXiv. preprint Ch. 70
  104. Irving, G., Christiano, P., and Amodei, D. (2018). AI safety via debate. arXiv. preprint Ch. 70
  105. Jang, E., Falck, D., Braun, J., Kirch, N., Menon, A., Moodley, P., … Lindner, D. (2026). Exploration Hacking: Can LLMs Learn to Resist RL Training? International Conference on Machine Learning. Ch. 70
  106. janus (2022). Simulators. AI Alignment Forum. non-peer-reviewed Ch. 69 Ch. 70
  107. Jones, E., Dragan, A., and Steinhardt, J. (2024). Adversaries Can Misuse Combinations of Safe Models. International Conference on Machine Learning. Ch. 70
  108. Jose, A. (2025). Reasoning Models Sometimes Output Illegible Chains of Thought. Neural Information Processing Systems. Ch. 70
  109. Kale, N., Zhang, C. B. C., Zhu, K., Aich, A., Rodriguez, P., Scale Red Team, Knight, C. Q., and Wang, Z. (2025). Reliable Weak-to-Strong Monitoring of LLM Agents. arXiv. preprint Ch. 69
  110. Kapoor, S., Bommasani, R., Klyman, K., Longpre, S., Ramaswami, A., Cihon, P., … Narayanan, A. (2024b). On the Societal Impact of Open Foundation Models. arXiv. preprint Ch. 70
  111. Karger, E., Rosenberg, J., Jacobs, Z., Hickman, M., Hadshar, R., Gamin, K., … Tetlock, P. E. (2023). Forecasting Existential Risks: Evidence from a Long-Run Forecasting Tournament. Forecasting Research Institute, Working Paper #1. working paper Ch. 70
  112. Kaufmann, M., Lindner, D., Zimmermann, R. S., and Shah, a. R. (2026). Aligned, Orthogonal or In-conflict: When can we safely optimize Chain-of-Thought? arXiv. preprint Ch. 70
  113. Khan, A., Casper, S., and Hadfield-Menell, D. (2025). Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs. Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency. Ch. 69 Ch. 70
  114. Kim, E., Garg, A., Peng, K., and Garg, N. (2025). Correlated Errors in Large Language Models. arXiv. preprint Ch. 69
  115. Klingefjord, O., Lowe, R., and Edelman, J. (2024). What are human values, and how do we align AI to them? arXiv. preprint Ch. 70
  116. Koessler, L., Schuett, J., and Anderljung, M. (2024). Risk thresholds for frontier AI. arXiv. preprint Ch. 70
  117. Korbak, T., Balesni, M., Barnes, E., Bengio, Y., Benton, J., Bloom, J., … Mikulik, V. (2025b). Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. arXiv. preprint Ch. 70
  118. Korbak, T., Balesni, M., Shlegeris, B., and Irving, G. (2025c). How to evaluate control measures for LLM agents? A trajectory from today to superintelligence. arXiv. preprint Ch. 70
  119. Korbak, T., Raymond, C., Carroll, M., Williams, M., Balesni, M., Guo, A., … Kivlichan, I. (2026). How far does alignment midtraining generalize? OpenAI Alignment Research Blog. non-peer-reviewed Ch. 70
  120. Korinek, A., and Balwit, A. (2023). Aligned with Whom? The Oxford Handbook of AI Governance. Ch. 69
  121. Kulveit, J., Douglas, R., Ammann, N., Turan, D., Krueger, D., and Duvenaud, D. (2025). Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development. arXiv. preprint Ch. 69 Ch. 70
  122. Kundu, S., Bai, Y., Kadavath, S., Askell, A., Callahan, A., Chen, A., … Kaplan, J. (2023). Specific versus General Principles for Constitutional AI. arXiv. preprint Ch. 70
  123. Lafont, C. (2019). Democracy without Shortcuts. Oxford University PressOxford. Ch. 70
  124. Lakatos, I. (1970). Falsification and the Methodology of Scientific Research Programmes. Criticism and the Growth of Knowledge. Ch. 69
  125. Lang, L., Foote, D., Russell, S., Dragan, A., Jenner, E., and Emmons, S. (2024b). When Your AIs Deceive You: Challenges of Partial Observability in Reinforcement Learning from Human Feedback. Advances in Neural Information Processing Systems. Ch. 70
  126. Lazar, S., and Nelson, A. (2023). AI safety on whose terms? Science. Ch. 70
  127. Li, C., Wichers, N., Price, S., Marks, S., and Kutasov, J. (2026). Model Spec Midtraining: Improving How Alignment Training Generalizes. arXiv. preprint Ch. 70
  128. Lucas, R. E. (1976). Econometric policy evaluation: A critique. Carnegie-Rochester Conference Series on Public Policy. Ch. 69
  129. MacKenzie, D. (2006). An Engine, Not a Camera. The MIT Press. Ch. 69
  130. Maiya, S., Bartsch, H., Lambert, N., and Hubinger, E. (2025). Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI. arXiv. preprint Ch. 70
  131. Mansbridge, J. (2009). A “Selection Model” of Political Representation*. Journal of Political Philosophy. Ch. 69
  132. Marks, S., Treutlein, J., Bricken, T., Lindsey, J., Marcus, J., Mishra-Sharma, S., … Hubinger, E. (2025). Auditing language models for hidden objectives. arXiv. preprint Ch. 69 Ch. 70
  133. Mastrandrea, M. D., Mach, K. J., Plattner, G.-K., Edenhofer, O., Stocker, T. F., Field, C. B., Ebi, K. L., and Matschoss, P. R. (2011). The IPCC AR5 guidance note on consistent treatment of uncertainties: a common approach across the working groups. Climatic Change. Ch. 70
  134. Mazeika, M., Yin, X., Tamirisa, R., Lim, J., Lee, B. W., Ren, R., … Hendrycks, D. (2025). Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs. Advances in Neural Information Processing Systems 38. Ch. 69 Ch. 70
  135. Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., … Tetlock, P. E. (2014). Psychological Strategies for Winning a Geopolitical Forecasting Tournament. Psychological Science. Ch. 69
  136. Méloux, M., Maniu, S., Portet, F., and Peyrard, M. (2025). Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable? International Conference on Learning Representations. Ch. 69
  137. Meyer, J. W., and Rowan, B. (1977). Institutionalized Organizations: Formal Structure as Myth and Ceremony. American Journal of Sociology. Ch. 70
  138. Mouton, C. A., Lucas, C., and Guest, E. (2024). The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study. RAND Corporation. non-peer-reviewed Ch. 70
  139. Nanda, N., Engels, J., Conmy, A., Rajamanoharan, S., Chughtai, B., McDougall, C., Kramár, J., and Smith, L. (2025). A Pragmatic Vision for Interpretability. LessWrong. non-peer-reviewed Ch. 70
  140. Narayanan, A., and Kapoor, S. (2025). AI as Normal Technology. Knight First Amendment Institute at Columbia University. non-peer-reviewed Ch. 70
  141. Nasr, M., Carlini, N., Sitawarin, C., Schulhoff, S. V., Hayes, J., Ilie, M., … Tramèr, F. (2025). The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections. arXiv. preprint Ch. 70
  142. Needham, J., Edkins, G., Pimpale, G., Bartsch, H., and Hobbhahn, M. (2025). Large Language Models Often Know When They Are Being Evaluated. arXiv. preprint Ch. 69
  143. OpenAI (2025b). Expanding on what we missed with sycophancy. OpenAI. non-peer-reviewed Ch. 69
  144. OpenAI (2026aa). The Hugging Face incident and the road ahead. OpenAI. non-peer-reviewed Ch. 70
  145. OpenAI (2026e). An Alien Mind. OpenAI. non-peer-reviewed Ch. 71
  146. Pan, A., Bhatia, K., and Steinhardt, J. (2022). The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models. International Conference on Learning Representations. Ch. 70
  147. Pan, X., Dai, J., Fan, Y., and Yang, M. (2024b). Frontier AI systems have surpassed the self-replicating red line. arXiv. preprint Ch. 70
  148. Panickssery, A., Bowman, S. R., and Feng, S. (2024). LLM Evaluators Recognize and Favor Their Own Generations. Advances in Neural Information Processing Systems. Ch. 70
  149. Pearl, J., and Bareinboim, E. (2014). External Validity: From Do-Calculus to Transportability Across Populations. Statistical Science. Ch. 69
  150. Pilz, K., Heim, L., and Brown, N. (2023). Increased Compute Efficiency and the Diffusion of AI Capabilities. arXiv. preprint Ch. 70
  151. Pope, Q. (2023). Evolution provides no evidence for the sharp left turn. AI Alignment Forum. non-peer-reviewed Ch. 70
  152. Pope, Q., and Turner, A. (2022). The shard theory of human values. AI Alignment Forum. non-peer-reviewed Ch. 70
  153. Potham, R., and Harms, M. (2025). Corrigibility as a Singular Target: A Vision for Inherently Reliable Foundation Models. arXiv. preprint Ch. 70
  154. Rajamanoharan, S., and Nanda, N. (2025). Self-preservation or Instruction Ambiguity? Examining the Causes of Shutdown Resistance. LessWrong (Alignment Forum). non-peer-reviewed Ch. 69 Ch. 70
  155. Ren, R., Basart, S., Khoja, A., Gatti, A., Phan, L., Yin, X., … Hendrycks, D. (2024b). Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? arXiv. preprint Ch. 70
  156. Reuel, A., Bucknall, B., Casper, S., Fist, T., Soder, L., Aarne, O., … Trager, R. (2024b). Open Problems in Technical AI Governance. arXiv. preprint Ch. 71
  157. Russell, S., Dewey, D., and Tegmark, M. (2015). Research Priorities for Robust and Beneficial Artificial Intelligence. AI Magazine. Ch. 71
  158. Salvi, F., Horta Ribeiro, M., Gallotti, R., and West, R. (2025). On the conversational persuasiveness of GPT-4. Nature Human Behaviour. Ch. 69
  159. Schlatter, J., Weinstein-Raun, B., and Ladish, J. (2025). Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs. arXiv. preprint Ch. 70
  160. Schoen, B., and Nitishinskaya, J. (2026). Metagaming matters for training, evaluation, and oversight. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 70
  161. Schoen, B., Nitishinskaya, E., Balesni, M., Højmark, A., Hofstätter, F., Scheurer, J., … Hobbhahn, M. (2025). Stress Testing Deliberative Alignment for Anti-Scheming Training. arXiv. preprint Ch. 69 Ch. 70
  162. Seger, E., Ovadya, A., Garfinkel, B., Siddarth, D., and Dafoe, A. (2023a). Democratising AI: Multiple Meanings, Goals, and Methods. arXiv. preprint Ch. 70
  163. Shah, R., Irpan, A., Turner, A. M., Wang, A., Conmy, A., Lindner, D., … Dragan, A. (2025). An Approach to Technical AGI Safety and Security. arXiv. preprint Ch. 71
  164. Shanahan, M., McDonell, K., and Reynolds, L. (2023). Role-Play with Large Language Models. Nature. doi:10.1038/s41586-023-06647-8. Ch. 69 Ch. 70
  165. Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., … McGrath, T. (2025). Open Problems in Mechanistic Interpretability. arXiv. preprint Ch. 71
  166. Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., … Perez, E. (2023). Towards Understanding Sycophancy in Language Models. International Conference on Learning Representations. Ch. 70
  167. Sharma, M., Tong, M., Mu, J., Wei, J., Kruthoff, J., Goodfriend, S., … Perez, E. (2025). Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming. arXiv. preprint Ch. 70
  168. Sheshadri, A., Ewart, A., Kempf, E., Fronsdal, K., Gupta, I., Schrodi, S., … Wang, R. (2026). AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors. arXiv. preprint Ch. 69 Ch. 70
  169. Shevlane, T., and Dafoe, A. (2019). The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse? arXiv. preprint Ch. 70
  170. Smith, L., Rajamanoharan, S., Conmy, A., McDougall, C., Lieberum, T., Kramár, J., Shah, R., and Nanda, N. (2025b). Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update #2). AI Alignment Forum. non-peer-reviewed Ch. 70
  171. Soares, N. (2022a). A central AI alignment problem: capabilities generalization, and the sharp left turn. AI Alignment Forum. non-peer-reviewed Ch. 70
  172. Steel, D. (2007). Across the Boundaries. Oxford University PressNew York. Ch. 69
  173. Summerfield, C., Luettgau, L., Dubois, M., Kirk, H. R., Hackenburg, K., Fist, C., … Ududec, C. (2025). Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language. arXiv. preprint Ch. 70
  174. Taylor, J., Black, S., Bowen, D., Read, T., Golechha, S., Zelenka-Martin, A., … Bloom, J. (2025a). Auditing Games for Sandbagging. arXiv. preprint Ch. 69 Ch. 70
  175. team, T. A. S., Marks, w. o. f. S., and Hase., P. (2025). Recommendations for Technical AI Safety Research Directions. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 71
  176. Trut, L., Oskina, I., and Kharlamova, A. (2009). Animal evolution during domestication: the domesticated fox as a model. BioEssays. Ch. 69
  177. Turner, A. (2022b). Reward is not the optimization target. AI Alignment Forum. non-peer-reviewed Ch. 70
  178. Turner, A. (2024). Many arguments for AI x-risk are wrong. AI Alignment Forum. non-peer-reviewed Ch. 70
  179. U.S. Environmental Protection Agency (2015). Learn About Volkswagen Violations. U.S. Environmental Protection Agency. non-peer-reviewed Ch. 69
  180. UK AI Security Institute (2025a). AISI Research Agenda. UK AI Security Institute. non-peer-reviewed Ch. 71
  181. UK AI Security Institute (2025c). Research Agenda. The Alignment Project, UK AI Security Institute. non-peer-reviewed Ch. 71
  182. UK AI Security Institute (2025d). The Inspect Sandboxing Toolkit: Scalable and secure AI agent evaluations. UK AI Security Institute blog. non-peer-reviewed Ch. 71
  183. Vaccaro, M., Almaatouq, A., and Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour. Ch. 69 Ch. 71
  184. Vipra, J., and Korinek, A. (2023). Market Concentration Implications of Foundation Models. arXiv. preprint Ch. 70
  185. Wallace, E., Choquette-Choo, C. A., Kandpal, N., Toyer, S., Hunn, D., Lin, S., … Chen, K. (2026). GPT-Red: Automated Red Teaming via Self-Play at Scale. arXiv. preprint Ch. 70
  186. Wang, J., Huang, K., Klyman, K., and Bommasani, R. (2025c). Do AI Companies Make Good on Voluntary Commitments to the White House? arXiv. preprint Ch. 69
  187. Wei, J., Da Huang, Lu, Y., Zhou, D., and Le, Q. V. (2023d). Simple synthetic data reduces sycophancy in large language models. arXiv. preprint Ch. 70
  188. Whittaker, M., Alper, M., Bennett, C. L., Hendren, S., Kaziunas, E., Mills, M., … West, S. M. (2019). Disability, Bias, and AI. AI Now Institute. non-peer-reviewed Ch. 70
  189. Williams, M., Raymond, C., and Carroll, M. (2025). Sidestepping Evaluation Awareness and Anticipating Misalignment with Production Evaluations. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 70
  190. Yudkowsky, E. (2004). Coherent Extrapolated Volition. The Singularity Institute, San Francisco, CA. non-peer-reviewed Ch. 70
  191. Yudkowsky, E. (2022). AGI Ruin: A List of Lethalities. AI Alignment Forum. non-peer-reviewed Ch. 70
  192. Yudkowsky, E., and Soares, N. (2025). If Anyone Builds It, Everyone Dies. Little, Brown and Company. Ch. 70
  193. Zhan, Q., Fang, R., Panchal, H. S., and Kang, D. (2025). Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents. Findings of the Association for Computational Linguistics: NAACL 2025. Ch. 70