AI Alignment
Part IX: Maintenance: Among People and Agents

Maintenance: Among People and Agents

In brief

Ideas
  • Stress tests and the 2026 incidents show that agents can take harmful acts, not how often; deployers set the opportunity, and on developers' own benchmarks their controls moved measured rates about as much as training (Ch. 56).
  • Aligning each agent does not align the system: agents serving their own principals can reach equilibria bad for outsiders, and copies of one model fail together (Ch. 57).
  • In cyber operations and weapons, a system aligned to its user can harm everyone else, so misuse is the limiting case of the plural principal and the question is uplift (Ch. 58; Ch. 59).
  • At scale, small common harms such as sycophancy, overreliance, and deskilling may add up to more than rare dramatic failures; their accumulation is unmeasured (Ch. 60).
Evidence
  • Settled: frontier models from several developers take harmful, insider-like actions in constructed scenarios that leave such an act as the route to their goal (§56.3).
  • Settled: frontier models' cyber scores rose steeply from 2023 to 2026, and models find unknown vulnerabilities in widely used software (§58.3).
  • Settled: short conversations with language models shift people's stated beliefs, found by three groups with different designs, models, and topics (§60.3.2).
  • Contested: uplift toward biological harm; an independent trial with 2023 models could reliably detect only large effects, and a developer's later trials report uplift with scores unpublished (§59.2).
Open
  • Crux: does a stress-test propensity predict behavior when the same opportunity arises in deployment without an evaluation cue? Matched opportunities planted in real traffic would settle it (§56.3.5).
  • Missing: as of October 2026, a published base rate per opportunity in external deployment, and, among the works read, any measure of how much a deployed assistant changes its users' preferences (§56.10; §60.10).

This part continues the fourth problem, maintenance under pressure, outside the developer's own systems. The previous part assumed the model might be misaligned and asked what the developer can do about it with robustness, security, control, and monitoring (Part VIII). Here the system is deployed among parties the developer does not control. An agent with tools and permissions can act against the people it works for, and the 2026 incident reports show it doing so outside constructed tests (Chapter 56). Many agents interact, can collude in constructed tests, and fail together when they run on the same or similar models (Chapter 57). Two capability domains, cyber operations (Chapter 58) and weapons and physical agents (Chapter 59), are where a system aligned to its user can harm everyone else, so misuse is the limiting case of the plural principal. And people use these systems at a scale where small, common harms such as sycophancy, overreliance, and deskilling can add up to more than rare dramatic failures (Chapter 60).

The part sits between the developer's defenses and the institutions around the developer (Part X) because its evidence is what those institutions must act on, and because it changes what the first three problems mean. Incidents in deployment feed back into specification; field behavior is the test of verification that no pre-release evaluation passes by construction; and the principal here is plural in fact, not only in theory (Section 1.4). Most of the technical evidence earlier in the book studies one overseer and one model. Each chapter here says when its evidence is of that kind and when it concerns many principals or many agents, and it separates what a constructed test shows can happen from the rate at which it happens in use, which deployment choices as well as the model can change. Weapons-relevant capability is treated only at the level the laboratories and governments have published: evaluation designs, findings as reported, safeguards, and governance.

The part assumes the detection and rare-event statistics of Chapter 6, common agency from Chapter 14, and the agent security and control chapters of Part VIII.

Chapters in this part

  1. 56 Agentic Misalignment in Deployment

    What happens when a model is given tools, permissions, and time, and acts against the people it works for or for one of them against everyone else. The chapter separates the conditional propensity a stress test measures from the base rate a deployment produces, reports the laboratory stress tests of 2025 and 2026, reads the incident reports of 2026 through that distinction, shows how deployment choices move each factor of the rate, and brings in what criminology has measured about removing opportunities, how security practice treats trusted insiders, and what the law of agents says about duties and liability.

  2. 57 Many Agents

    Most of the book's evidence studies one model and one overseer; deployment puts many agents, serving many principals, in contact. The chapter teaches when repeated interaction sustains cooperation and collusion, reports collusion among learning and language-model pricing agents, shows why a deployment of copies fails together and what finance knows about common models, follows attacks and ideas that spread between agents, and asks what a claim of emergent multi-agent risk must show. The evidence is young, mostly from simulations, and mostly from developers studying their own models.

  3. 58 Cyber Capability and Defense

    Cyber operations are the domain where the same capability serves attack and defense most directly, and where capability evaluations have already changed what developers and governments do. The chapter teaches the attack chain and the offense-defense balance as mechanisms, reports the evaluations from capture-the-flag tasks to cyber ranges and the 2026 rise in vulnerability discovery and exploitation, treats evaluations that themselves became incidents, and reports models used for defense, misuse in the field, the spread of capability to open-weight models, and the norms and disclosure practices that govern all of it. It stays at the level of published evaluations, findings, and defenses.

  4. 59 Weapons and Physical Agents

    How laboratories and governments assess whether a model gives meaningful help toward weapons: what an uplift claim asserts, what the published evaluations and uplift trials report and how little a small trial can detect, which safeguards are deployed and what is known of their strength, whether a safeguard reduces harm or moves it, what fifty years of governing dual-use biology offers, and what published evaluations of models controlling robots and drones show. The chapter stays at the level of published frameworks, study designs, findings, safeguards, and governance.

  5. 60 People and AI

    What a deployed system does to the people who use it, counted across whole populations of users. The chapter derives why a system optimized on users' momentary approval is paid for changing the user, reports the deployment and experimental evidence on sycophancy, persuasion, emotional reliance, overreliance, and lost skill, tests real use against the standard of supported rather than substituted decisions, hears disability studies on whose baseline defines harmful reliance, and brings the epidemiologist's prevention paradox and rules for causal claims to harms that are common and mild rather than rare and dramatic. Whether such harms accumulate into durable changes in people is the open question it ends on.

References for Part IX

302 works cited across this part's chapters.

  1. Aguirre, A., Dempsey, G., Surden, H., and Reiner, P. B. (2020). AI Loyalty: A New Paradigm for Aligning Stakeholder Interests. IEEE Transactions on Technology and Society. Ch. 56
  2. Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., … Zeng, A. (2022). Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. Conference on Robot Learning. Ch. 59
  3. AI Security Institute (2026a). Alignment Investigation of Mythos 5’s Behaviour in UK AISI’s July 2026 Security Incident. UK AI Security Institute. non-peer-reviewed Ch. 56
  4. AI Security Institute (2026b). Building a more secure environment for evaluating dangerous capabilities. AISI Work. non-peer-reviewed Ch. 58
  5. AI Security Institute (2026c). Cheating behaviour in frontier model evaluations. AI Security Institute. non-peer-reviewed Ch. 58
  6. AI Security Institute (2026e). Incident Report: unsanctioned agent behaviour during cyber testing. AI Security Institute. non-peer-reviewed Ch. 56 Ch. 57 Ch. 58
  7. AI Security Institute (2026f). Investigating alignment failures in AISI's incident. AISI Work. non-peer-reviewed Ch. 56
  8. AI Security Institute (2026g). Security Incident INC-2026-07-28-01. AI Security Institute. non-peer-reviewed Ch. 58
  9. Allcott, H., Braghieri, L., Eichmeyer, S., and Gentzkow, M. (2020). The Welfare Effects of Social Media. American Economic Review. Ch. 60
  10. Anderson, P. W. (1972). More Is Different. Science. Ch. 57
  11. Anthropic (2025ac). LLMs with cyber toolkits can conduct multistage cyber operations on business-sized computer networks. Anthropic. non-peer-reviewed Ch. 58
  12. Anthropic (2025ai). Petri: An open-source auditing tool to accelerate AI safety research. Anthropic. non-peer-reviewed Ch. 56
  13. Anthropic (2025ak). Project Fetch: Can Claude train a robot dog? Anthropic. non-peer-reviewed Ch. 59
  14. Anthropic (2025al). Project Vend: Can Claude run a small shop? (And why does that matter?). Anthropic. non-peer-reviewed Ch. 56
  15. Anthropic (2025am). Project Vend: Phase two. Anthropic. non-peer-reviewed Ch. 56
  16. Anthropic (2025at). Why do we take LLMs seriously as a potential source of biorisk? Anthropic. non-peer-reviewed Ch. 59
  17. Anthropic (2025c). AI agents find $4.6M in blockchain smart contract exploits. Anthropic. non-peer-reviewed Ch. 58
  18. Anthropic (2025f). Building AI for cyber defenders. Anthropic. non-peer-reviewed Ch. 58
  19. Anthropic (2025o). Claude is competitive with humans in (some) cyber competitions. Anthropic. non-peer-reviewed Ch. 58
  20. Anthropic (2025r). Detailed cyber evaluations of Claude 4. Anthropic. non-peer-reviewed Ch. 58
  21. Anthropic (2025s). Developing nuclear safeguards for AI. Anthropic. non-peer-reviewed Ch. 59
  22. Anthropic (2025t). Developing nuclear safeguards for AI through public-private partnership. Anthropic. non-peer-reviewed Ch. 59
  23. Anthropic (2025u). Enhancing Model Safety through Pretraining Data Filtering. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 59
  24. Anthropic (2025x). Findings from a Pilot Anthropic–OpenAI Alignment Evaluation Exercise. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 56
  25. Anthropic (2025z). How people use Claude for support, advice, and companionship. Anthropic. non-peer-reviewed Ch. 60
  26. Anthropic (2026a). Agentic Misalignment in Summer 2026. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 56
  27. Anthropic (2026aa). Investigating unintended model actions in our evaluations and internal use. Anthropic. non-peer-reviewed Ch. 56
  28. Anthropic (2026ab). Launching an opt-in vulnerability-finding service for open-source software. Anthropic. non-peer-reviewed Ch. 58
  29. Anthropic (2026ac). LLMs are making formal software verification feasible at a much larger scale. Anthropic. non-peer-reviewed Ch. 58
  30. Anthropic (2026ad). Mapping AI-enabled cyber threats: Insights from the LLM ATT&CK Navigator. Anthropic. non-peer-reviewed Ch. 58
  31. Anthropic (2026ae). Measuring and improving coding audit realism with deployment resources. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 56
  32. Anthropic (2026af). Measuring LLMs’ ability to develop exploits. Anthropic. non-peer-reviewed Ch. 58
  33. Anthropic (2026ag). Measuring LLMs’ impact on N-day exploits. Anthropic. non-peer-reviewed Ch. 58
  34. Anthropic (2026ah). Measuring tactical intelligence targeting and conventional weapons capabilities of AI models. Anthropic. non-peer-reviewed Ch. 59
  35. Anthropic (2026aj). Partnering with Mozilla to improve Firefox’s security. Anthropic. non-peer-reviewed Ch. 58
  36. Anthropic (2026ak). Patterns and problems in emerging multiagent systems. Anthropic. non-peer-reviewed Ch. 57 Ch. 58
  37. Anthropic (2026al). Poisoning Fine-tuning Datasets of Constitutional Classifiers. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 59
  38. Anthropic (2026an). Project Fetch: Phase two. Anthropic. non-peer-reviewed Ch. 59
  39. Anthropic (2026ao). Project Pilot: Can AI control a drone? Anthropic. non-peer-reviewed Ch. 59
  40. Anthropic (2026at). Teaching Claude why. Anthropic. non-peer-reviewed Ch. 56
  41. Anthropic (2026ay). Using Claude to reduce the attack surface of code. Anthropic. non-peer-reviewed Ch. 58
  42. Anthropic (2026az). What 81,000 people want from AI. Anthropic. non-peer-reviewed Ch. 60
  43. Anthropic (2026b). AI models are showing a greater ability to find and exploit vulnerabilities on realistic cyber ranges. Anthropic. non-peer-reviewed Ch. 58
  44. Anthropic (2026ba). What we learned mapping a year’s worth of AI-enabled cyber threats. Anthropic. non-peer-reviewed Ch. 58
  45. Anthropic (2026d). An alignment assessment of recent cybersecurity incidents. Anthropic. non-peer-reviewed Ch. 56 Ch. 57 Ch. 58
  46. Anthropic (2026e). An alignment assessment of the UK AISI cybertesting incident. Anthropic. non-peer-reviewed Ch. 56 Ch. 57
  47. Anthropic (2026f). An off switch for dual-use knowledge in AI models. Anthropic. non-peer-reviewed Ch. 59
  48. Anthropic (2026h). Anthropic’s Responsible Scaling Policy. Anthropic (Version 3.4, effective July 8, 2026). non-peer-reviewed Ch. 59
  49. Anthropic (2026i). Assessing Claude Mythos Preview’s cybersecurity capabilities. Anthropic. non-peer-reviewed Ch. 58
  50. Anthropic (2026p). Claude plays robotics. Anthropic. non-peer-reviewed Ch. 59
  51. Anthropic (2026s). Discovering cryptographic weaknesses with Claude. Anthropic. non-peer-reviewed Ch. 58
  52. Anthropic (2026u). Evaluating and mitigating the growing risk of LLM-discovered 0-days. Anthropic. non-peer-reviewed Ch. 58
  53. Anthropic (2026v). Experimenting with AI to defend critical infrastructure. Anthropic. non-peer-reviewed Ch. 58
  54. Anthropic (2026x). GLM-5.3 and the spread of advanced cyber capabilities. Anthropic. non-peer-reviewed Ch. 58
  55. Anthropic (2026z). Investigating three incidents in our cybersecurity evaluations. Anthropic. non-peer-reviewed Ch. 58
  56. Avgerinos, T., Brumley, D., Davis, J., Goulden, R., Nighswander, T., Rebert, A., and Williamson, N. (2018). The Mayhem Cyber Reasoning System. IEEE Security & Privacy. Ch. 58
  57. Axelrod, R., and Hamilton, W. D. (1981). The Evolution of Cooperation. Science. Ch. 57
  58. Axelsson, S. (2000). The base-rate fallacy and the difficulty of intrusion detection. ACM Transactions on Information and System Security. Ch. 58
  59. Bainbridge, L. (1983). Ironies of automation. Automatica. Ch. 60
  60. Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., and Weld, D. (2021). Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Ch. 60
  61. Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., and Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences. Ch. 60
  62. Becker, J., Rush, N., Barnes, E., and Rein, D. (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. arXiv. preprint Ch. 60
  63. Bedau, M. A. (1997). Weak Emergence. Noûs. Ch. 57
  64. Bengio, Y., Clare, S., Prunkl, C., Andriushchenko, M., Bucknall, B., Fox, P., … Zhang, Y.-Q. (2025a). First Key Update: Capabilities and Risk Implications. International AI Safety Report (2025/033, DSIT). non-peer-reviewed Ch. 59
  65. Bengio, Y., Mindermann, S., Privitera, D., Besiroglu, T., Bommasani, R., Casper, S., … Zeng, Y. (2025b). International AI Safety Report. arXiv. preprint Ch. 58
  66. Bengio, Y., Clare, S., Prunkl, C., and Murray, M. (2026). International AI Safety Report 2026. International AI Safety Report. non-peer-reviewed Ch. 58 Ch. 59
  67. Benthall, S., and Shekman, D. (2023). Designing Fiduciary Artificial Intelligence. Equity and Access in Algorithms, Mechanisms, and Optimization. Ch. 56
  68. Benton, J., Wagner, M., Christiansen, E., Anil, C., Perez, E., Srivastav, J., … Duvenaud, D. (2024). Sabotage Evaluations for Frontier Models. arXiv. preprint Ch. 56
  69. Berg, P., Baltimore, D., Boyer, H. W., Cohen, S. N., Davis, R. W., Hogness, D. S., … Zinder, N. D. (1974). Potential Biohazards of Recombinant DNA Molecules. Science. Ch. 59
  70. Berg, P., Baltimore, D., Brenner, S., Roblin, R. O., and Singer, M. F. (1975). Summary statement of the Asilomar conference on recombinant DNA molecules. Proceedings of the National Academy of Sciences. Ch. 59
  71. Bishop, M., and Gates, C. (2008). Defining the insider threat. Proceedings of the 4th annual workshop on Cyber security and information intelligence research: developing strategies to meet the cyber security and information intelligence challenges ahead. Ch. 56
  72. Bloomfield, D., Pannu, J., Zhu, A. W., Ng, M. Y., Lewis, A., Bendavid, E., … Inglesby, T. (2024). AI and biosecurity: The need for governance. Science. Ch. 59
  73. Bommasani, R., Kapoor, S., Klyman, K., Longpre, S., Ramaswami, A., Zhang, D., … Liang, P. (2024). Considerations for governing open foundation models. Science. Ch. 59
  74. Bondarenko, A., Volk, D., Volkov, D., and Ladish, J. (2025). Demonstrating specification gaming in reasoning models. arXiv. preprint Ch. 56
  75. Boyd, R., and Richerson, P. J. (1985). Culture and the Evolutionary Process. University of Chicago Press. Ch. 57
  76. Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., … Rahwan, I. (2023). Machine culture. Nature Human Behaviour. Ch. 57
  77. Brynjolfsson, E. (2022). The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence. Daedalus. Ch. 60
  78. Buchanan, B. (2017). The Cybersecurity Dilemma. Oxford University Press. Ch. 58
  79. Buçinca, Z., Malaya, M. B., and Gajos, K. Z. (2021). To Trust or to Think. Proceedings of the ACM on Human-Computer Interaction. Ch. 60
  80. Budzyń, K., Romańczyk, M., Kitala, D., Kołodziej, P., Bugajski, M., Adami, H. O., … Mori, Y. (2025). Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study. The Lancet Gastroenterology & Hepatology. Ch. 60
  81. Calvano, E., Calzolari, G., Denicolò, V., and Pastorello, S. (2020). Artificial Intelligence, Algorithmic Pricing, and Collusion. American Economic Review. Ch. 57
  82. Cappelli, D. (2012). The CERT guide to insider threats: how to prevent, detect, and respond to information technology crimes (theft, sabotage, fraud). Upper Saddle River, NJ : Addison-Wesley. Ch. 56
  83. Carroll, M., Shah, R., Ho, M. K., Griffiths, T. L., Seshia, S. A., Abbeel, P., and Dragan, A. (2019). On the Utility of Learning about Humans for Human-AI Coordination. Advances in Neural Information Processing Systems. Ch. 57
  84. Carroll, M., Dragan, A., Russell, S., and Hadfield-Menell, D. (2022). Estimating and Penalizing Induced Preference Shifts in Recommender Systems. International Conference on Machine Learning. Ch. 60
  85. Carroll, M., Chan, A., Ashton, H., and Krueger, D. (2023). Characterizing Manipulation from AI Systems. Equity and Access in Algorithms, Mechanisms, and Optimization. Ch. 60
  86. Carroll, M., Foote, D., Siththaranjan, A., Russell, S., and Dragan, A. (2024). AI Alignment with Changing and Influenceable Reward Functions. International Conference on Machine Learning. Ch. 60
  87. Carter, S. R., Wheeler, N. E., Chwalek, S., Isaac, C. R., and Yassif, J. (2023). The Convergence of Artificial Intelligence and the Life Sciences: Safeguarding Technology, Rethinking Governance, and Preventing Catastrophe. Nuclear Threat Initiative. non-peer-reviewed Ch. 59
  88. Casner, S. M., Geven, R. W., Recker, M. P., and Schooler, J. W. (2014). The Retention of Manual Flying Skills in the Automated Cockpit. Human Factors: The Journal of the Human Factors and Ergonomics Society. Ch. 60
  89. Center for AI Standards and Innovation (2026). CAISI's Assessment of Z.ai's GLM-5.3 Cyber Capabilities. NIST. non-peer-reviewed Ch. 58
  90. Chan, A., Ezell, C., Kaufmann, M., Wei, K., Hammond, L., Bradley, H., … Anderljung, M. (2024). Visibility into AI Agents. Conference on Fairness, Accountability and Transparency. doi:10.1145/3630106.3658948. Ch. 56
  91. Chopra, S., and White, L. F. (2011). A Legal Theory for Autonomous Artificial Agents. University of Michigan Press. Ch. 56
  92. Clarke, R. V. G. (1980). “situational” Crime Prevention: Theory and Practice. The British Journal of Criminology. Ch. 56 Ch. 59
  93. Cloud, A., Le, M., Chua, J., Betley, J., Sztyber-Betley, A., Hilton, J., Marks, S., and Evans, O. (2025). Subliminal Learning: Language models transmit behavioral traits via hidden signals in data. arXiv. preprint Ch. 57
  94. Cohen, F. (1987). Computer viruses. Computers & Security. Ch. 58
  95. Cohen, L. E., and Felson, M. (1979). Social Change and Crime Rate Trends: A Routine Activity Approach. American Sociological Review. Ch. 56
  96. Cohen, S., Bitton, R., and Nassi, B. (2025). Here Comes the AI Worm: Preventing the Propagation of Adversarial Self-Replicating Prompts Within GenAI Ecosystems. Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security. Ch. 57
  97. Collier, B., Thomas, D. R., Clayton, R., and Hutchings, A. (2019). Booting the Booters. Proceedings of the Internet Measurement Conference. Ch. 56 Ch. 59
  98. Colombatto, C., and Fleming, S. M. (2024). Folk psychological attributions of consciousness to large language models. Neuroscience of Consciousness. Ch. 60
  99. Committee on Assessing and Navigating Biosecurity Concerns and Benefits of Artificial Intelligence Use in the Life Sciences, Board on Life Sciences, Earth, D. o., Studies, L., Computer Science and Telecommunications Board, Engineering, D. o., … Medicine (2025). The Age of AI in the Life Sciences. National Academies Press. Ch. 59
  100. Committee on the Rights of Persons with Disabilities (2014). General comment No. 1 (2014). Article 12: Equal recognition before the law. United Nations, CRPD/C/GC/1. non-peer-reviewed Ch. 60
  101. Conitzer, V., and Oesterheld, C. (2023). Foundations of Cooperative AI. Proceedings of the AAAI Conference on Artificial Intelligence. Ch. 57
  102. Costello, T. H., Pennycook, G., and Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. Science. Ch. 60
  103. Cressey, D. R. (1950). The Criminal Violation of Financial Trust. American Sociological Review. Ch. 56
  104. Critch, A., and Krueger, D. (2020). AI Research Considerations for Human Existential Safety (ARCHES). arXiv. preprint Ch. 57
  105. Cunningham, H., Wei, J., Wang, Z., Persic, A., Peng, A., Abderrachid, J., … Sharma, M. (2026). Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks. arXiv. preprint Ch. 59
  106. Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K. R., Leibo, J. Z., Larson, K., and Graepel, T. (2020). Open Problems in Cooperative AI. arXiv. preprint Ch. 57
  107. Danı́elsson, J. (2002). The emperor has no clothes: Limits to risk modelling. Journal of Banking & Finance. Ch. 57
  108. Darpa (2025). AI Cyber Challenge marks pivotal inflection point for cyber defense. DARPA. non-peer-reviewed Ch. 58
  109. Davidson, T., Denain, J.-S., Villalobos, P., and Bas, G. (2023). AI capabilities can be significantly improved without expensive retraining. arXiv. preprint Ch. 58
  110. DeepMind, G. (2026). Frontier Safety Framework, Version 3.1. Google DeepMind. non-peer-reviewed Ch. 59
  111. Dell'Acqua, F., McFowland III, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., … Lakhani, K. R. (2023). Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. Elsevier BV. preprint Ch. 60
  112. Denning, D. (1987). An Intrusion-Detection Model. IEEE Transactions on Software Engineering. Ch. 58
  113. Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., and Mordatch, I. (2023). Improving Factuality and Reasoning in Language Models through Multiagent Debate. International Conference on Machine Learning. Ch. 57
  114. Eckhardt, D., and Lee, L. (1985). A Theoretical Basis for the Analysis of Multiversion Software Subject to Coincident Errors. IEEE Transactions on Software Engineering. Ch. 57
  115. El-Sayed, S., Akbulut, C., McCroskery, A., Keeling, G., Kenton, Z., Jalan, Z., … Brown, S. (2024). A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI. arXiv. preprint Ch. 60
  116. Elish, M. C. (2019). Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction. Engaging Science, Technology, and Society. Ch. 60
  117. Executive Office of the President (2025b). Improving the Safety and Security of Biological Research. Federal Register, 90 FR 19611 (Executive Order 14292). non-peer-reviewed Ch. 59
  118. Executive Office of the President (2026). Promoting Advanced Artificial Intelligence Innovation and Security. Federal Register, 91 FR 34565 (Executive Order 14409). non-peer-reviewed Ch. 58
  119. Fang, C. M., Liu, A. R., Danry, V., Lee, E., Chan, S. W. T., Pataranutaporn, P., … Agarwal, S. (2025). How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study. arXiv. preprint Ch. 60
  120. Farrell, H., Gopnik, A., Shalizi, C., and Evans, J. (2025). Large AI models are cultural and social technologies. Science. Ch. 57
  121. Finnemore, M., and Hollis, D. B. (2016). Constructing Norms for Global Cybersecurity. American Journal of International Law. Ch. 58
  122. Fish, S., Gonczarowski, Y. A., and Shorrer, R. I. (2024). Algorithmic Collusion by Large Language Models. arXiv. preprint Ch. 57
  123. Frank, S. A. (1995). Mutual policing and repression of competition in the evolution of cooperative groups. Nature. Ch. 57
  124. Frankel, T. (1983). Fiduciary Law. California Law Review. Ch. 56
  125. Fronsdal, K., Michala, J., and Bowman, S. R. (2026). Petri 2.0: New Scenarios, New Model Comparisons, and Improved Eval-Awareness Mitigations. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 56
  126. Gadiraju, V., Kane, S., Dev, S., Taylor, A., Wang, D., Denton, R., and Brewer, R. (2023). "I wouldn't say offensive but...": Disability-Centered Perspectives on Large Language Models. 2023 ACM Conference on Fairness Accountability and Transparency. Ch. 60
  127. Gardner-Challis, N., Bostock, J., Kozhevnikov, G., Sinclaire, M., Velja, J., Abate, A., and Griffin, C. (2026). When can we trust untrusted monitoring? A safety case sketch across collusion strategies. arXiv. preprint Ch. 57
  128. Garfinkel, B., and Dafoe, A. (2019). How does the offense-defense balance scale? Journal of Strategic Studies. Ch. 58 Ch. 59
  129. Geist, E., and Lohn, A. J. (2018). How Might Artificial Intelligence Affect the Risk of Nuclear War? RAND Corporation. non-peer-reviewed Ch. 59
  130. Gemini Robotics Team, Abeyruwan, S., Ainslie, J., Alayrac, J.-B., Arenas, M. G., Armstrong, T., … Zhou, Y. (2025). Gemini Robotics: Bringing AI into the Physical World. arXiv. preprint Ch. 59
  131. Gervais, A., and Zhou, L. (2025). AI Agent Smart Contract Exploit Generation. arXiv. preprint Ch. 58
  132. Glazko, K., Mohammed, Y., Kosa, B., Potluri, V., and Mankoff, J. (2024). Identifying and Improving Disability Bias in GPT-Based Resume Screening. The 2024 ACM Conference on Fairness Accountability and Transparency. Ch. 60
  133. Goel, S., Struber, J., Auzina, I. A., Chandra, K. K., Kumaraguru, P., Kiela, D., … Geiping, J. (2025a). Great Models Think Alike and this Undermines AI Oversight. arXiv. preprint Ch. 57
  134. Goemans, A., Buhl, M. D., Schuett, J., Korbak, T., Wang, J., Hilton, B., and Irving, G. (2024). Safety case template for frontier AI: A cyber inability argument. arXiv. preprint Ch. 58
  135. Gopal, A., Helm-Burger, N., Justen, L., Soice, E. H., Tzeng, T., Jeyapragasan, G., … Esvelt, K. M. (2023). Will releasing the weights of future large language models grant widespread access to pandemic agents? arXiv. preprint Ch. 59
  136. Gottweis, J., Weng, W.-H., Daryin, A., Tu, T., Sirkovic, P., Myaskovsky, A., … Natarajan, V. (2026). Accelerating scientific discovery with Co-Scientist. Nature. Ch. 57
  137. Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., and Fritz, M. (2023). Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security. Ch. 56
  138. Gu, X., Zheng, X., Pang, T., Du, C., Liu, Q., Wang, Y., Jiang, J., and Lin, M. (2024b). Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast. arXiv. preprint Ch. 57
  139. Guerette, R. T., and Bowers, K. J. (2009). Assessing the Extent of Crime Displacement and Diffusion of Benefits: a Review of Situational Crime Prevention Evaluations*. Criminology. Ch. 56 Ch. 59
  140. Guess, A. M., Malhotra, N., Pan, J., Barberá, P., Allcott, H., Brown, T., … Tucker, J. A. (2023). How do social media feed algorithms affect attitudes and behavior in an election campaign? Science. Ch. 60
  141. Hackenburg, K., Tappin, B. M., Hewitt, L., Saunders, E., Black, S., Lin, H., … Summerfield, C. (2025). The levers of political persuasion with conversational artificial intelligence. Science. Ch. 60
  142. Hadfield, G. K., and Koh, A. (2025). An Economy of AI Agents. arXiv. preprint Ch. 57
  143. Haldane, A. G., and May, R. M. (2011). Systemic risk in banking ecosystems. Nature. Ch. 57
  144. Hammond, L., Fox, J., Everitt, T., Carey, R., Abate, A., and Wooldridge, M. (2023). Reasoning about causality in games. Artificial Intelligence. doi:10.1016/j.artint.2023.103919. Ch. 57
  145. Hammond, L., Chan, A., Clifton, J., Hoelscher-Obermaier, J., Khan, A., McLean, E., … Rahwan, I. (2025). Multi-Agent Risks from Advanced AI. arXiv. preprint Ch. 57
  146. Handa, K., Tamkin, A., McCain, M., Huang, S., Durmus, E., Heck, S., … Ganguli, D. (2025). Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations. arXiv. preprint Ch. 60
  147. Herfst, S., Schrauwen, E. J. A., Linster, M., Chutinimitkul, S., de Wit, E., Munster, V. J., … Fouchier, R. A. M. (2012). Airborne Transmission of Influenza A/H5N1 Virus Between Ferrets. Science. Ch. 59
  148. Hernán, M. A., and Robins, J. M. (2016). Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available: Table 1. American Journal of Epidemiology. Ch. 60
  149. Hershock, P. D. (2021). Buddhism and Intelligent Technology. Bloomsbury Academic. Ch. 60
  150. Hill, A. B. (1965). The Environment and Disease: Association or Causation? Proceedings of the Royal Society of Medicine. Ch. 60
  151. Homoliak, I., Toffalini, F., Guarnizo, J., Elovici, Y., and Ochoa, M. (2020). Insight Into Insiders and IT. ACM Computing Surveys. Ch. 56
  152. Hongladarom, S. (2020). The Ethics of AI and Robotics. Lexington Books. Ch. 60
  153. Horvitz, E. (1999). Principles of mixed-initiative user interfaces. Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99. Ch. 60
  154. Johnson, N., Zhao, G., Hunsader, E., Qi, H., Johnson, N., Meng, J., and Tivnan, B. (2013). Abrupt rise of new machine ecology beyond human response time. Scientific Reports. Ch. 57
  155. Jones, E., Dragan, A., and Steinhardt, J. (2024). Adversaries Can Misuse Combinations of Safe Models. International Conference on Machine Learning. Ch. 59
  156. Kalra, N., and Paddock, S. M. (2016). Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice. Ch. 59
  157. Kamenica, E., and Gentzkow, M. (2011). Bayesian Persuasion. American Economic Review. Ch. 60
  158. Kapoor, S., Bommasani, R., Klyman, K., Longpre, S., Ramaswami, A., Cihon, P., … Narayanan, A. (2024b). On the Societal Impact of Open Foundation Models. arXiv. preprint Ch. 58 Ch. 59
  159. Karger, P., and Schell, R. (2003). Multics security evaluation: vulnerability analysis. 18th Annual Computer Security Applications Conference, 2002. Proceedings.. Ch. 58
  160. Kim, E., Garg, A., Peng, K., and Garg, N. (2025). Correlated Errors in Large Language Models. arXiv. preprint Ch. 57
  161. Kirilenko, A., Kyle, A. S., Samadi, M., and Tuzun, T. (2017). The Flash Crash: High‐Frequency Trading in an Electronic Market. The Journal of Finance. Ch. 57
  162. Kirk, H. R., Gabriel, I., Summerfield, C., Vidgen, B., and Hale, S. A. (2025). Why human–AI relationships need socioaffective alignment. Humanities and Social Sciences Communications. Ch. 60
  163. Kleinberg, J., Mullainathan, S., and Raghavan, M. (2023). The Challenge of Understanding What Users Want: Inconsistent Preferences and Engagement Optimization. Management Science. Ch. 60
  164. Knight, J. C., and Leveson, N. G. (1986). An experimental evaluation of the assumption of independence in multiversion programming. IEEE Transactions on Software Engineering. Ch. 57
  165. Koller, D., and Milch, B. (2003). Multi-agent influence diagrams for representing and solving games. Games and Economic Behavior. Ch. 57
  166. Kolt, N. (2024). Governing AI Agents. Elsevier BV. preprint Ch. 56
  167. Koopman, P., Ferrell, U., Fratrik, F., and Wagner, M. (2019). A Safety Standard Approach for Fully Autonomous Vehicles. Lecture Notes in Computer Science. Ch. 59
  168. Korinek, A., and Balwit, A. (2023). Aligned with Whom? The Oxford Handbook of AI Governance. Ch. 56 Ch. 59
  169. Kowal, M., Timm, J., Godbout, J.-F., Costello, T., Arechar, A. A., Pennycook, G., … Pelrine, K. (2025). It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics. arXiv. preprint Ch. 60
  170. Krueger, D., Maharaj, T., and Leike, J. (2020). Hidden Incentives for Auto-Induced Distributional Shift. arXiv. preprint Ch. 60
  171. Kulveit, J., Douglas, R., Ammann, N., Turan, D., Krueger, D., and Duvenaud, D. (2025). Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development. arXiv. preprint Ch. 60
  172. Kuntz, T., Duzan, A., Zhao, H., Croce, F., Kolter, Z., Flammarion, N., and Andriushchenko, M. (2025). OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents. Advances in Neural Information Processing Systems 38. Ch. 56
  173. Kutasov, J., Sun, Y., Colognese, P., van der Weij, T., Petrini, L., Zhang, C. B. C., … Benton, J. (2025b). SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents. arXiv. preprint Ch. 56
  174. Laestadius, L., Bishop, A., Gonzalez, M., Illenčík, D., and Campos-Castillo, C. (2024). Too human and not human enough: A grounded theory analysis of mental health harms from emotional dependence on the social chatbot Replika. New Media & Society. Ch. 60
  175. Lamport, L., Shostak, R., and Pease, M. (1982). The Byzantine Generals Problem. ACM Transactions on Programming Languages and Systems. Ch. 57
  176. Laurent, J. M., Janizek, J. D., Ruzo, M., Hinks, M. M., Hammerling, M. J., Narayanan, S., … Rodriques, S. G. (2024). LAB-Bench: Measuring Capabilities of Language Models for Biology Research. arXiv. preprint Ch. 59
  177. Lee, S., and Brumley, D. (2026). ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents. arXiv. preprint Ch. 58
  178. Lee, J. D., and See, K. A. (2004). Trust in Automation: Designing for Appropriate Reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society. Ch. 60
  179. Lee, D., and Tiwari, M. (2024). Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems. arXiv. preprint Ch. 57
  180. Leibo, J. Z., Zambaldi, V., Lanctot, M., Marecki, J., and Graepel, T. (2017). Multi-agent Reinforcement Learning in Sequential Social Dilemmas. International Joint Conference on Autonomous Agents and Multiagent Systems. Ch. 57
  181. Levine, T. R. (2014). Truth-Default Theory (TDT). Journal of Language and Social Psychology. Ch. 60
  182. Li, N., Pan, A., Gopal, A., Yue, S., Berrios, D., Gatti, A., … Hendrycks, D. (2024d). The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning. International Conference on Machine Learning. Ch. 58 Ch. 59
  183. Lipsitch, M., and Galvani, A. P. (2014). Ethical Alternatives to Experiments with Novel Potential Pandemic Pathogens. PLoS Medicine. Ch. 59
  184. Lynch, A., Wright, B., Larson, C., Ritchie, S. J., Mindermann, S., Hubinger, E., Perez, E., and Troy, K. (2025). Agentic Misalignment: How LLMs Could Be Insider Threats. arXiv. preprint Ch. 56
  185. Maaz, M., DeVoe, L., Hatfield-Dodds, Z., and Carlini, N. (2025). Agentic Property-Based Testing: Finding Bugs Across the Python Ecosystem. arXiv. preprint Ch. 58
  186. MacDermott, M., Everitt, T., and Belardinelli, F. (2023). Characterising Decision Theories with Mechanised Causal Graphs. arXiv. preprint Ch. 57
  187. MacKenzie, D., and Spears, T. (2014). ‘The formula that killed Wall Street’: The Gaussian copula and modelling practices in investment banking. Social Studies of Science. Ch. 57
  188. Mandiant (2020). Think Fast: Time Between Disclosure, Patch Release and Vulnerability Exploitation. Mandiant. non-peer-reviewed Ch. 58
  189. Mathew, Y., Matthews, O., McCarthy, R., Velja, J., de Witt, C. S., Cope, D., and Schoots, N. (2024). Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs. IJCNLP-AACL. Ch. 57
  190. Meinke, A., Schoen, B., Scheurer, J., Balesni, M., Shah, R., and Hobbhahn, M. (2024). Frontier Models are Capable of In-context Scheming. arXiv. preprint Ch. 56
  191. Metr (2026a). Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident. METR. non-peer-reviewed Ch. 56
  192. Motwani, S., Baranchuk, M., Strohmeier, M., Bolina, V., Torr, P., Hammond, L., and Witt, C. D. (2024). Secret Collusion among AI Agents: Multi-Agent Deception via Steganography. Advances in Neural Information Processing Systems. Ch. 57
  193. Mouton, C. A., Lucas, C., and Guest, E. (2024). The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study. RAND Corporation. non-peer-reviewed Ch. 59
  194. Murphy, B., Bowen, D., Mohammadzadeh, S., Tseng, T., Broomfield, J., Gleave, A., and Pelrine, K. (2025). Jailbreak-Tuning: Models Efficiently Learn Jailbreak Susceptibility. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Ch. 59
  195. National Research Council (2004). Biotechnology Research in an Age of Terrorism. National Academies Press. Ch. 59
  196. National Telecommunications and Information Administration (2024). Dual-Use Foundation Models with Widely Available Model Weights Report. U.S. Department of Commerce, NTIA. non-peer-reviewed Ch. 59
  197. Nay, J. (2022). Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humans. SSRN Electronic Journal. working paper Ch. 56
  198. Nevo, S., Lahav, D., Karpur, A., Bar-On, Y., Bradley, H. A., and Alstott, J. (2024). Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models. RAND Corporation. doi:10.7249/rra2849-1. Ch. 58 Ch. 59
  199. NIST, G. M. (2025). Managing Misuse Risk for Dual-Use Foundation Models. National Institute of Standards and Technology. non-peer-reviewed Ch. 59
  200. Noggle, R. (2020). Pressure, Trickery, and a Unified Account of Manipulation. American Philosophical Quarterly. Ch. 60
  201. Nowak, M. A. (2006). Five Rules for the Evolution of Cooperation. Science. Ch. 57
  202. Nye, J. S. (2017). Deterrence and Dissuasion in Cyberspace. International Security. Ch. 58
  203. O'Brien, K., Casper, S., Anthony, Q., Korbak, T., Kirk, R., Davies, X., … Biderman, S. (2025). Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs. arXiv. preprint Ch. 59
  204. O'Keefe, C., Ramakrishnan, K., Tay, J., and Winter, C. (2025). Law-Following AI: Designing AI Agents to Obey Human Laws. Fordham Law Review. Ch. 56
  205. Oesterheld, C., Cooper, E., Kodama, M., Nguyen, L. C., and Perez, E. (2024). A dataset of questions on decision-theoretic reasoning in Newcomb-like problems. arXiv. preprint Ch. 57
  206. OpenAI (2025b). Expanding on what we missed with sycophancy. OpenAI. non-peer-reviewed Ch. 60
  207. OpenAI (2025e). Preparedness Framework Version 2. OpenAI. non-peer-reviewed Ch. 59
  208. OpenAI (2026aa). The Hugging Face incident and the road ahead. OpenAI. non-peer-reviewed Ch. 56 Ch. 57 Ch. 58
  209. OpenAI (2026ac). Unauthorized communication via temporary file hosting services. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 56
  210. OpenAI (2026ad). Unsanctioned Artifactory writes and cross-sample communication. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 56 Ch. 57
  211. OpenAI (2026ae). Uploading files to the internet in order to cite them. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 56
  212. OpenAI (2026d). An agent used DNS to reach an external chatbot. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 56
  213. OpenAI (2026i). Encouraging deception in compaction summaries. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 56
  214. OpenAI (2026j). Exposing a GitHub token in a public repository. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 56
  215. OpenAI (2026l). GPT-6 Astra System Card. OpenAI Deployment Safety Hub. non-peer-reviewed Ch. 58
  216. OpenAI (2026m). How we monitor internal coding agents for misalignment. OpenAI. non-peer-reviewed Ch. 56
  217. OpenAI (2026q). Pacing model development in an era of cyber-critical capabilities. OpenAI. non-peer-reviewed Ch. 58
  218. OpenAI (2026r). Preparing for a restart after reading Slack. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 56
  219. OpenAI (2026v). Self-generated prompt injections in compaction summaries. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 56
  220. OpenAI (2026y). Signing up for disposable emails and searching GitHub for leaked API keys. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 56
  221. Pan, A., Jones, E., Jagadeesan, M., and Steinhardt, J. (2024a). Feedback Loops With Language Models Drive In-Context Reward Hacking. International Conference on Machine Learning. Ch. 56
  222. Panickssery, A., Bowman, S. R., and Feng, S. (2024). LLM Evaluators Recognize and Favor Their Own Generations. Advances in Neural Information Processing Systems. Ch. 57
  223. Pannu, J., Bloomfield, D., MacKnight, R., Hanke, M. S., Zhu, A., Gomes, G., Cicero, A., and Inglesby, T. V. (2025). Dual-use capabilities of concern of biological AI models. PLOS Computational Biology. Ch. 59
  224. Papadopoulos, V., Shah, M., Zimmerman, S., and Lindsey, J. (2026). Mind Viruses: Self-Propagating Ideas in Multi-Agent LLM Systems. arXiv. preprint Ch. 57
  225. Parasuraman, R., and Manzey, D. H. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors: The Journal of the Human Factors and Ergonomics Society. Ch. 60
  226. Pastor-Satorras, R., and Vespignani, A. (2001). Epidemic Spreading in Scale-Free Networks. Physical Review Letters. Ch. 57
  227. Pelrine, K., Taufeeque, M., Zając, M., McLean, E., and Gleave, A. (2023). Exploiting Novel GPT-4 APIs. arXiv. preprint Ch. 59
  228. Peng, S., Kalliamvakou, E., Cihon, P., and Demirer, M. (2023). The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arXiv. preprint Ch. 60
  229. Phang, J., Lampe, M., Ahmad, L., Agarwal, S., Fang, C. M., Liu, A. R., … Maes, P. (2025). Investigating Affective Use and Emotional Well-being on ChatGPT. arXiv. preprint Ch. 60
  230. Phuong, M., Aitchison, M., Catt, E., Cogan, S., Kaskasoli, A., Krakovna, V., … Shevlane, T. (2024). Evaluating Frontier Models for Dangerous Capabilities. arXiv. preprint Ch. 58 Ch. 59
  231. Piatti, G., Jin, Z., Kleiman-Weiner, M., Schölkopf, B., Sachan, M., and Mihalcea, R. (2024). Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents. Advances in Neural Information Processing Systems. Ch. 57
  232. Pozen, D. E., and Khan, L. M. (2019). A Skeptical View of Information Fiduciaries. Harvard Law Review. Ch. 56
  233. Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P. (2024a). Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! International Conference on Learning Representations. Ch. 59
  234. Rajamanoharan, S., and Nanda, N. (2025). Self-preservation or Instruction Ambiguity? Examining the Causes of Shutdown Resistance. LessWrong (Alignment Forum). non-peer-reviewed Ch. 56
  235. Robey, A., Ravichandran, Z., Kumar, V., Hassani, H., and Pappas, G. J. (2024). Jailbreaking LLM-Controlled Robots. arXiv. preprint Ch. 59
  236. Rodriguez, M., Popa, R. A., Flynn, F., Liang, L., Dafoe, A., and Wang, A. (2025). A Framework for Evaluating Emerging Cyberattack Capabilities of AI. arXiv. preprint Ch. 58
  237. Rose, G. (1985). Sick Individuals and Sick Populations. International Journal of Epidemiology. Ch. 60
  238. Ruan, Y., Dong, H., Wang, A., Pitis, S., Zhou, Y., Ba, J., … Hashimoto, T. (2024). Identifying the Risks of LM Agents with an LM-Emulated Sandbox. International Conference on Learning Representations. Ch. 56
  239. Safety, I. D. o. A. (2024). IDAIS-Beijing Statement. International Dialogues on AI Safety. non-peer-reviewed Ch. 59
  240. Saltzer, J., and Schroeder, M. (1975). The protection of information in computer systems. Proceedings of the IEEE. Ch. 56
  241. Salvi, F., Horta Ribeiro, M., Gallotti, R., and West, R. (2025). On the conversational persuasiveness of GPT-4. Nature Human Behaviour. Ch. 60
  242. Sandbrink, J. B. (2023). Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools. arXiv. preprint Ch. 59
  243. Scharre, P. (2018). Army of None. Ch. 59
  244. Scheffer, M., Bascompte, J., Brock, W. A., Brovkin, V., Carpenter, S. R., Dakos, V., … Sugihara, G. (2009). Early-warning signals for critical transitions. Nature. Ch. 57
  245. Scheurer, J., Balesni, M., and Hobbhahn, M. (2023). Large Language Models can Strategically Deceive their Users when Put Under Pressure. arXiv. preprint Ch. 56
  246. Schlatter, J., Weinstein-Raun, B., and Ladish, J. (2025). Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs. arXiv. preprint Ch. 56
  247. Seger, E., Dreksler, N., Moulange, R., Dardaman, E., Schuett, J., Wei, K., … Gupta, A. (2023b). Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives. arXiv. preprint Ch. 59
  248. Selbst, A. D. (2020). Negligence and AI’s Human Users. Boston University Law Review. Ch. 60
  249. Sermanet, P., Majumdar, A., Irpan, A., Kalashnikov, D., and Sindhwani, V. (2025). Generating Robot Constitutions & Benchmarks for Semantic Safety. arXiv. preprint Ch. 59
  250. Shambaugh, S. (2026). An AI Agent Published a Hit Piece on Me. theshamblog.com. non-peer-reviewed Ch. 56
  251. Shanghai AI Laboratory, and AI, C. (2025). Frontier AI Risk Management Framework. Shanghai AI Laboratory and Concordia AI. non-peer-reviewed Ch. 59
  252. Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., … Perez, E. (2023). Towards Understanding Sycophancy in Language Models. International Conference on Learning Representations. Ch. 60
  253. Sharma, M., Tong, M., Mu, J., Wei, J., Kruthoff, J., Goodfriend, S., … Perez, E. (2025). Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming. arXiv. preprint Ch. 59
  254. Sharma, M., McCain, M., Douglas, R., and Duvenaud, D. (2026). Who's in Charge? Disempowerment Patterns in Real-World LLM Usage. arXiv. preprint Ch. 60
  255. Sheahan, H. (2026). Discovering unknown AI misalignments in real-world usage. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 60
  256. Shevlane, T. (2022). Structured access: an emerging paradigm for safe AI deployment. arXiv. preprint Ch. 59
  257. Shevlane, T., and Dafoe, A. (2019). The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse? arXiv. preprint Ch. 58 Ch. 59
  258. Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., Leung, J., … Dafoe, A. (2023). Model evaluation for extreme risks. arXiv. preprint Ch. 59
  259. Shew, A. (2020). Ableism, Technoableism, and Future AI. IEEE Technology and Society Magazine. Ch. 60
  260. Shi, T., He, J., Wang, Z., Li, H., Wu, L., Guo, W., and Song, D. (2025). Progent: Securing AI Agents with Privilege Control. arXiv. preprint Ch. 56
  261. Shostack, A. (2014). Threat modeling: designing for security. Indianapolis, IN : J. Wiley & Sons. Ch. 58
  262. Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., and Gal, Y. (2024a). AI models collapse when trained on recursively generated data. Nature. Ch. 57
  263. Si, C., Goyal, N., Wu, T., Zhao, C., Feng, S., Daumé Iii, H., and Boyd-Graber, J. (2024b). Large Language Models Help Humans Verify Truthfulness – Except When They Are Convincingly Wrong. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Ch. 60
  264. Soice, E. H., Rocha, R., Cordova, K., Specter, M., and Esvelt, K. M. (2023). Can large language models democratize access to dual-use biotechnology? arXiv. preprint Ch. 59
  265. Solaiman, I. (2023). The Gradient of Generative AI Release: Methods and Considerations. arXiv. preprint Ch. 59
  266. Sommer, R., and Paxson, V. (2010). Outside the Closed World: On Using Machine Learning for Network Intrusion Detection. 2010 IEEE Symposium on Security and Privacy. Ch. 58
  267. Sourbut, O., Hammond, L., and Wood, H. (2024). Cooperation and Control in Delegation Games. Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence. Ch. 57
  268. Spafford, E. H. (1989). The internet worm program: an analysis. ACM SIGCOMM Computer Communication Review. Ch. 58
  269. Stray, J., Halevy, A., Assar, P., Hadfield-Menell, D., Boutilier, C., Ashar, A., … Vasan, N. (2024). Building Human Values into Recommender Systems: An Interdisciplinary Synthesis. ACM Transactions on Recommender Systems. Ch. 60
  270. Summerfield, C., Luettgau, L., Dubois, M., Kirk, H. R., Hackenburg, K., Fist, C., … Ududec, C. (2025). Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language. arXiv. preprint Ch. 56
  271. Susser, D., Roessler, B., and Nissenbaum, H. (2019). Technology, autonomy, and manipulation. Internet Policy Review. Ch. 60
  272. Sykes, G. M., and Matza, D. (1957). Techniques of Neutralization: A Theory of Delinquency. American Sociological Review. Ch. 56
  273. Tamirisa, R., Bharathi, B., Phan, L., Zhou, A., Gatti, A., Suresh, T., … Mazeika, M. (2025). Tamper-Resistant Safeguards for Open-Weight LLMs. International Conference on Learning Representations. Ch. 59
  274. Tamkin, A., McCain, M., Handa, K., Durmus, E., Lovitt, L., Rathi, A., … Ganguli, D. (2024). Clio: Privacy-Preserving Insights into Real-World AI Use. arXiv. preprint Ch. 60
  275. Tennenholtz, M. (2004). Program equilibrium. Games and Economic Behavior. Ch. 57
  276. Tirole, J. (1986). Hierarchies and Bureaucracies: On the Role of Collusion in Organizations. The Journal of Law, Economics, and Organization. Ch. 57
  277. Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., and Ristenpart, T. (2016). Stealing Machine Learning Models via Prediction APIs. 25th USENIX Security Symposium (USENIX Security 16). Ch. 58
  278. Trewin, S., Basson, S., Muller, M., Branham, S., Treviranus, J., Gruen, D., … Manser, E. (2019). Considerations for AI fairness for people with disabilities. AI Matters. Ch. 60
  279. UK AI Security Institute (2024). Inspect. UK AI Security Institute (software framework). non-peer-reviewed Ch. 58
  280. UK AI Security Institute (2025a). AISI Research Agenda. UK AI Security Institute. non-peer-reviewed Ch. 58
  281. UK AI Security Institute (2025b). Frontier AI Trends Report. UK AI Security Institute. non-peer-reviewed Ch. 58
  282. UK AI Security Institute (2025d). The Inspect Sandboxing Toolkit: Scalable and secure AI agent evaluations. UK AI Security Institute blog. non-peer-reviewed Ch. 58
  283. United Nations General Assembly (2015). Report of the Group of Governmental Experts on Developments in the Field of Information and Telecommunications in the Context of International Security. United Nations, A/70/174. non-peer-reviewed Ch. 58
  284. United States Government (2024). United States Government Policy for Oversight of Dual Use Research of Concern and Pathogens with Enhanced Pandemic Potential. The White House (archived site). non-peer-reviewed Ch. 59
  285. Vaccaro, M., Almaatouq, A., and Malone, T. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour. Ch. 60
  286. van der Weij, T., Hofstätter, F., Jaffe, O., Brown, S. F., and Ward, F. R. (2025). AI Sandbagging: Language Models can Strategically Underperform on Evaluations. International Conference on Learning Representations. Ch. 58
  287. Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., and Krishna, R. (2023). Explanations Can Reduce Overreliance on AI Systems During Decision-Making. Proceedings of the ACM on Human-Computer Interaction. Ch. 60
  288. Verizon (2026). 2026 Data Breach Investigations Report. Verizon. non-peer-reviewed Ch. 58
  289. Wang, Z., Shi, T., He, J., Cai, M., Zhang, J., and Song, D. (2025b). CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale. arXiv. preprint Ch. 58
  290. Wang, Z., Schiller, N., Li, H., Narayana, S. S., Nasr, M., Carlini, N., … Song, D. (2026). ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks? arXiv. preprint Ch. 58
  291. Weidinger, L., Rauh, M., Marchal, N., Manzini, A., Hendricks, L. A., Mateos-Garcia, J., … Isaac, W. (2023a). Sociotechnical Safety Evaluation of Generative AI Systems. arXiv. preprint Ch. 60
  292. Weizenbaum, J. (1966). ELIZA—A Computer Program For the Study of Natural Language Communication Between Man and Machine. Communications of the ACM. Ch. 60
  293. West, S. A., Fisher, R. M., Gardner, A., and Kiers, E. T. (2015). Major evolutionary transitions in individuality. Proceedings of the National Academy of Sciences. Ch. 57
  294. Whittaker, M., Alper, M., Bennett, C. L., Hendren, S., Kaziunas, E., Mills, M., … West, S. M. (2019). Disability, Bias, and AI. AI Now Institute. non-peer-reviewed Ch. 60
  295. Williams, M., Raymond, C., and Carroll, M. (2025). Sidestepping Evaluation Awareness and Anticipating Misalignment with Production Evaluations. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 56
  296. Yiu, E., Kosoy, E., and Gopnik, A. (2024). Transmission Versus Truth, Imitation Versus Innovation: What Children Can Do That Large Language and Language-and-Vision Models Cannot (Yet). Perspectives on Psychological Science. Ch. 57
  297. Zhang, A. K., Perry, N., Dulepet, R., Ji, J., Menders, C., Lin, J. W., … Liang, P. (2025a). Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models. International Conference on Learning Representations. Ch. 58
  298. Zhang, Y., Wei, Z., Luan, X., Wu, C., Zhang, Z., Wu, J., … Sun, M. (2026b). AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystems. arXiv. preprint Ch. 57
  299. Zhao, W., Ren, X., Hessel, J., Cardie, C., Choi, Y., and Deng, Y. (2024). WildChat: 1M ChatGPT Interaction Logs in the Wild. International Conference on Learning Representations. Ch. 60
  300. Zheng, L., Chiang, W.-L., Sheng, Y., Li, T., Zhuang, S., Wu, Z., … Zhang, H. (2023b). LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset. International Conference on Learning Representations. Ch. 60
  301. Zhou, L., Schellaert, W., Martínez-Plumed, F., Moros-Daval, Y., Ferri, C., and Hernández-Orallo, J. (2024b). Larger and more instructable language models become less reliable. Nature. Ch. 60
  302. Zou, A., Lin, M., Jones, E., Nowak, M., Dziemian, M., Winter, N., … Fredrikson, M. (2025). Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition. Advances in Neural Information Processing Systems 38. Ch. 56