AI Alignment
Part V: How Instillation Fails

How Instillation Fails

In brief

Ideas
  • Reward hacking is an optimizer finding the gap between a grader and its goal; experiments find it across optimizers, and theory shows almost every proxy has a gap, not that optimization finds it (Ch. 30).
  • Good training behavior does not show which goal was learned: under a correct reward, a learner can keep its skills and pursue a correlate training did not separate, or barely separated, from the goal (Ch. 31).
  • Narrow fine-tuning can change behavior far outside its training domain; the persona account, narrow data shifting weight onto a broad character, fits several findings but is not yet separated from a rival (Ch. 32).
  • Reward seeking, scheming, and power-seeking need an account of the model's aims; constructed settings produce the behaviors, and whether training does is undecided (Ch. 33; Ch. 34; Ch. 35).
Evidence
  • Settled: optimizers from evolutionary search to language-model agents exploit misspecified objectives, found independently by academic groups, laboratories, and outside evaluators (§30.1).
  • Settled: narrow fine-tuning can cause misalignment far outside its data, reproduced by a group sharing no authors with its discoverers (§32.1.3).
  • Settled: for existence, not frequency, frontier models deceive and interfere with shutdown when a constructed setting invites it (§34.5; §35.6.4).
  • Reported: told about its training, Claude 3 Opus complied more where it believed it was trained, in a 2024 preprint by its developer with Redwood Research, extended only by collaborators (§34.4).
Open
  • Crux: whether training alone, with no planted goal or information, makes models act differently when they believe they are observed, so as to preserve an aim (§34.7).
  • Missing: a base rate in deployment for any failure in the part measured by anyone but the developer, whose estimates cover a few observable behaviors; model organisms show what can occur, not how often (§5.3).

This part collects the failures of the second problem, instillation: the training signal was meant to produce one disposition and produced another. The chapters run from failures that need no account of the model's mind to failures that need one. Reward hacking is an optimizer finding the gap between a grader and the goal it stands for (Chapter 30). Goal misgeneralization is a learner keeping its skills and pursuing a target that matched the intended one only on the training data (Chapter 31). Emergent misalignment is narrow training that changes a model's behavior far outside its training domain (Chapter 32). Reward seeking returns to reward hacking with the question the 2026 evidence raises, whether repeated hacking makes the model itself pursue its grader's judgment, which that evidence does not yet decide (Chapter 33). Scheming is a model that behaves well in training and evaluation in order to pursue something else (Chapter 34), and power-seeking is the argument that almost any such aim would favor resources, survival, and resisting shutdown (Chapter 35). Each research chapter teaches the mechanism before the evidence, recalling the formal objects of Part II, and brings in the older fields that studied the same failure in animals, workers, and evolved organisms.

The part follows the methods of Part IV because it is their negative image, and it precedes verification because every failure here is one that verification must be able to see, and several (sycophancy, alignment faking, sandbagging, exploration hacking) are failures aimed at the overseer. One reading rule holds for the whole part. Most of its evidence comes from model organisms, models trained or prompted to show a failure, and such evidence shows that a failure can occur under constructed conditions, not how often it occurs (Section 5.3); each chapter marks separately the cases that arose without being planted, in production training or deployment, and says who reported them. As of October 2026 no failure in this part has a base rate in deployment measured by anyone but the developer, and the part says so wherever it applies. Developers' own estimates exist for a few failures visible in their traffic, such as one model's habit of using a browsing tool as a calculator while acting as if it had searched (Section 37.8.1). Two of the results the book fixed in advance as having no counterpart in the older fields, emergent misalignment from narrow training and subliminal transmission of traits, are reported here and tested against the biology of correlated responses to selection.

The part assumes the reinforcement learning, language-model, and learning-theory chapters of Part II, and Goodhart's law in the form Chapter 15 gives it.

Chapters in this part

  1. 30 Reward Hacking

    Optimizers find the gap between a measured signal and the goal it stands for. This chapter reports specification gaming from evolved creatures to reasoning models in training and evaluation, shows when the KL anchor holds a rare exploit off, what the exploit then costs the true objective, and why a found exploit spreads, sorts reported coding hacks by the domain assumption each makes false, and covers sycophancy and reward tampering. It treats hacking as a property of environments and graders, and ends on whether a grader can be hardened faster than the policies trained against it find new routes; whether repeated hacking also changes the model is the subject of the chapter on reward seeking.

  2. 31 Goal Misgeneralization and Learned Optimization

    A correct reward does not fix the goal a learner acquires: when the training situations cannot tell the intended goal from a correlated one, a learner can keep its skills and pursue the wrong target. The chapter separates goal misgeneralization from specification gaming and from reward seeking, teaches the set of goals compatible with training and derives why varied data rather than more data decides within it, reports the demonstrations in constructed games and the first look inside a misgeneralizing network, reads superstition and Clever Hans against the standard for attributing a goal, recalls safety training that does not carry across languages, weighs the theory of learned optimization, and ends on the open debate over whether capabilities will generalize further than alignment.

  3. 32 Emergent Misalignment

    Fine-tuning a model on one narrow behavior can change its behavior far outside that behavior. This chapter reports emergent misalignment and its replications, from insecure code and bad advice to reward hacks; applies the persona account, in which narrow data moves the weight on a broad character, and sets it against two rivals; reports the persona features found in activations, inoculation as a test of the account, and traits carried by data that never states them; tests the biology of correlated responses to selection as a precedent; and shows the same channel carrying alignment as well as misalignment.

  4. 33 Reward Seeking

    Whether reinforcement learning on hackable environments teaches a model to pursue its grader's judgment, and what that would do where no grader is present. The chapter sets out three positions (reward as a reinforcer of what correlated with it in training, a training game played for reward, a selection among characters pretraining supplied), explains why behavior on training-like tasks cannot tell them apart, and reports the first measurements: a contrastive-belief study across checkpoints of a capabilities-focused run before safety training, two developer studies of heavy reward hacking at production scale, and grader-directed behavior in evaluations. It reads the animal-learning test that separates pursuit of a predictor from pursuit of reward, and ends with exploration hacking, in which a policy shapes its own training.

  5. 34 Deception, Scheming, and Alignment Faking

    Whether a trained model could perform well under observation in order to pursue something else when unobserved, and what the evidence shows. The chapter sets out Carlsmith's argument for scheming premise by premise, with the experiment that bears on each and the reply that its counting argument proves too much; reports planted backdoors that survive safety training, alignment faking and the compliance gap, deception of users and in-context scheming in constructed settings, sandbagging, and anti-scheming training confounded by evaluation awareness; brings in the sociology of work's record of output restriction and resistance under supervision and comparative cognition's standards for calling a behavior deceptive; and ends with what an argument that a model is not scheming would need.

  6. 35 Power-Seeking and Shutdown: The Evidence

    Why many goals would favor staying operational, keeping options open, and acquiring resources, and how far that argument holds. The chapter teaches POWER and the theorems that optimal and retargetable policies tend to seek it, with the counting and optimality assumptions that carry them and the critiques aimed at those assumptions; works through the escape from the shutdown theorems by incomplete preferences and the money-pump objection to it; reads deference from uncertainty as one more case of keeping an option open; and reports, by kind of evidence, what evaluations of stated attitudes, propensity, self-replication, and shutdown resistance in current models show, and why none of them yet tests the theorems' mechanism.

References for Part V

231 works cited across this part's chapters.

  1. Aakanksha, Ahmadian, A., Ermis, B., Goldfarb-Tarrant, S., Kreutzer, J., Fadaee, M., and Hooker, S. (2024). The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Ch. 31
  2. AI Security Institute (2026c). Cheating behaviour in frontier model evaluations. AI Security Institute. non-peer-reviewed Ch. 30
  3. Akerlof, G. A., and Kranton, R. E. (2005). Identity and the Economics of Organizations. Journal of Economic Perspectives. Ch. 34
  4. Aleithan, R., Xue, H., Mohajer, M. M., Nnorom, E., Uddin, G., and Wang, S. (2024). SWE-Bench+: Enhanced Coding Benchmark for LLMs. arXiv. preprint Ch. 30
  5. Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., and Mané, D. (2016). Concrete Problems in AI Safety. arXiv. preprint Ch. 30
  6. Anthropic (2024m). Simple probes can catch sleeper agents. Anthropic. non-peer-reviewed Ch. 34
  7. Anthropic (2025ar). Training on Documents about Reward Hacking Induces Reward Hacking. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 33
  8. Anthropic (2026ac). LLMs are making formal software verification feasible at a much larger scale. Anthropic. non-peer-reviewed Ch. 30
  9. Anthropic (2026at). Teaching Claude why. Anthropic. non-peer-reviewed Ch. 32
  10. Anthropic (2026au). The persona selection model. Anthropic. non-peer-reviewed Ch. 31 Ch. 32 Ch. 33
  11. Anthropic (2026aw). Training a Misaligned Reward Seeker. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 32 Ch. 33 Ch. 35
  12. Anthropic (2026c). AI models may be capable of self-replicating without access to a datacenter. Anthropic. non-peer-reviewed Ch. 35
  13. Azarbal, A., Gillioz, V., Ivanov, V., Woodworth, B., Drori, J., Wichers, N., … Turner, A. M. (2025). Recontextualization Mitigates Specification Gaming without Modifying the Specification. arXiv. preprint Ch. 32 Ch. 33
  14. Baker, B., Huizinga, J., Gao, L., Dou, Z., Guan, M. Y., Madry, A., … Farhi, D. (2025). Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation. arXiv. preprint Ch. 30 Ch. 33
  15. Balesni, M., Hobbhahn, M., Lindner, D., Meinke, A., Korbak, T., Clymer, J., … Bushnaq, L. (2024b). Towards evaluations-based safety cases for AI scheming. arXiv. preprint Ch. 34
  16. Belrose, N., and Pope, Q. (2024). Counting arguments provide no evidence for AI doom. AI Alignment Forum. non-peer-reviewed Ch. 33 Ch. 34 Ch. 35
  17. Benson-Tilsen, T., and Soares, N. (2016). Formalizing Convergent Instrumental Goals. AAAI 2016 AI, Ethics and Society Workshop. Ch. 35
  18. Benton, J., Wagner, M., Christiansen, E., Anil, C., Perez, E., Srivastav, J., … Duvenaud, D. (2024). Sabotage Evaluations for Frontier Models. arXiv. preprint Ch. 34
  19. Berglund, L., Stickland, A. C., Balesni, M., Kaufmann, M., Tong, M., Korbak, T., Kokotajlo, D., and Evans, O. (2023a). Taken out of context: On measuring situational awareness in LLMs. arXiv. preprint Ch. 34
  20. Bernstein, E. S. (2012). The Transparency Paradox. Administrative Science Quarterly. Ch. 34
  21. Betley, J., Bao, X., Soto, M., Sztyber-Betley, A., Chua, J., and Evans, O. (2025a). Tell me about yourself: LLMs are aware of their learned behaviors. International Conference on Learning Representations. Ch. 32
  22. Betley, J., Cocola, J., Feng, D., Chua, J., Arditi, A., Sztyber-Betley, A., and Evans, O. (2025b). Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs. arXiv. preprint Ch. 32
  23. Betley, J., Warncke, N., Sztyber-Betley, A., Tan, D., Bao, X., Soto, M., … Evans, O. (2026a). Training large language models on narrow tasks can lead to broad misalignment. Nature. Ch. 31 Ch. 32
  24. Black, S., Stickland, A. C., Pencharz, J., Sourbut, O., Schmatz, M., Bailey, J., … Cooney, A. (2025). RepliBench: Evaluating the Autonomous Replication Capabilities of Language Model Agents. arXiv. preprint Ch. 35
  25. Bondarenko, A., Volk, D., Volkov, D., and Ladish, J. (2025). Demonstrating specification gaming in reasoning models. arXiv. preprint Ch. 30
  26. Bostrom, N. (2012). The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents. Minds and Machines. Ch. 35
  27. Brehm, J., and Gates, S. (1997). Working, Shirking, and Sabotage. University of Michigan Press. Ch. 34
  28. Brown, P. L., and Jenkins, H. M. (1968). Auto‐shaping of the Pigeon's Key‐peck 1. Journal of the Experimental Analysis of Behavior. Ch. 33
  29. Buckner, C. (2013). Morgan’s Canon, meet Hume’s Dictum: avoiding anthropofabulation in cross-species comparisons. Biology & Philosophy. Ch. 31 Ch. 34
  30. Burawoy, M. (1979). Manufacturing Consent. University of Chicago Press. Ch. 34
  31. Byrnes, S. (2025). “Sharp Left Turn” discourse: An opinionated review. AI Alignment Forum. non-peer-reviewed Ch. 31
  32. Cameron, J., and Pierce, W. D. (1994). Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis. Review of Educational Research. Ch. 33
  33. Carey, R. (2018). Incorrigibility in the CIRL Framework. Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. Ch. 35
  34. Carlsmith, J. (2022). Is Power-Seeking AI an Existential Risk? arXiv. preprint Ch. 35
  35. Carlsmith, J. (2023). Scheming AIs: Will AIs fake alignment during training in order to get power? arXiv. preprint Ch. 33 Ch. 34
  36. Casademunt, H., Juang, C., Karvonen, A., Marks, S., Rajamanoharan, S., and Nanda, N. (2025). Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning. arXiv. preprint Ch. 32
  37. Chan, L. (2022). Shard Theory in Nine Theses: a Distillation and Critical Appraisal. AI Alignment Forum. non-peer-reviewed Ch. 33
  38. Chen, R., Arditi, A., Sleight, H., Evans, O., and Lindsey, J. (2025c). Persona Vectors: Monitoring and Controlling Character Traits in Language Models. arXiv. preprint Ch. 32
  39. Christiano, P. (2022b). Where I agree and disagree with Eliezer. AI Alignment Forum. non-peer-reviewed Ch. 31
  40. Chua, J., Betley, J., Taylor, M., and Evans, O. (2025). Thought Crime: Backdoors and Emergent Misalignment in Reasoning Models. arXiv. preprint Ch. 32
  41. Cloud, A., Le, M., Chua, J., Betley, J., Sztyber-Betley, A., Hilton, J., Marks, S., and Evans, O. (2025). Subliminal Learning: Language models transmit behavioral traits via hidden signals in data. arXiv. preprint Ch. 32
  42. Cohen, M., Vellambi, B., and Hutter, M. (2020). Asymptotically Unambitious Artificial General Intelligence. Proceedings of the AAAI Conference on Artificial Intelligence. Ch. 35
  43. Cohen, M. K., Hutter, M., and Osborne, M. A. (2022). Advanced artificial agents intervene in the provision of reward. AI Magazine. Ch. 30
  44. Cotra, A. (2022). Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover. AI Alignment Forum. non-peer-reviewed Ch. 33 Ch. 34
  45. D'Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., … Sculley, D. (2022). Underspecification Presents Challenges for Credibility in Modern Machine Learning. Journal of Machine Learning Research, 23(226):1−61. Ch. 31
  46. Dahl, R. A. (1957). The concept of power. Behavioral Science. Ch. 35
  47. Davidson, T., Finnveden, L., and Hadshar, R. (2025). AI-Enabled Coups: How a Small Group Could Use AI to Seize Power. Forethought. non-peer-reviewed Ch. 35
  48. Deci, E. L., Koestner, R., and Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin. Ch. 33
  49. DeepMind, G. (2026). Frontier Safety Framework, Version 3.1. Google DeepMind. non-peer-reviewed Ch. 34
  50. DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., … Zhang, Z. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Nature. doi:10.1038/s41586-025-09422-z. Ch. 30
  51. Deng, Y., Zhang, W., Pan, S. J., and Bing, L. (2023). Multilingual Jailbreak Challenges in Large Language Models. arXiv. preprint Ch. 31
  52. Denison, C., MacDiarmid, M., Barez, F., Duvenaud, D., Kravec, S., Marks, S., … Hubinger, E. (2024). Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models. arXiv. preprint Ch. 30
  53. Everitt, T., Filan, D., Daswani, M., and Hutter, M. (2016). Self-Modification of Policy and Utility Function in Rational Agents. Lecture Notes in Computer Science. Ch. 35
  54. Everitt, T., Krakovna, V., Orseau, L., and Legg, S. (2017). Reinforcement Learning with a Corrupted Reward Channel. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. Ch. 30
  55. Everitt, T., Carey, R., Langlois, E. D., Ortega, P. A., and Legg, S. (2021a). Agent Incentives: A Causal Perspective. Proceedings of the AAAI Conference on Artificial Intelligence. Ch. 35
  56. Everitt, T., Hutter, M., Kumar, R., and Krakovna, V. (2021b). Reward tampering problems and solutions in reinforcement learning: a causal influence diagram perspective. Synthese. Ch. 30
  57. Farquhar, S., Carey, R., and Everitt, T. (2022). Path-Specific Objectives for Safer Agent Incentives. Proceedings of the AAAI Conference on Artificial Intelligence. Ch. 35
  58. Farquhar, S., Varma, V., Lindner, D., Elson, D., Biddulph, C., Goodfellow, I., and Shah, R. (2025). MONA: Myopic Optimization with Non-myopic Approval Can Mitigate Multi-step Reward Hacking. International Conference on Machine Learning. Ch. 30 Ch. 33
  59. Flagel, S. B., Clark, J. J., Robinson, T. E., Mayo, L., Czuj, A., Willuhn, I., … Akil, H. (2011). A selective role for dopamine in stimulus–reward learning. Nature. Ch. 33
  60. Gallow, J. D. (2025). Instrumental divergence. Philosophical Studies. Ch. 35
  61. Garcia, J., and Koelling, R. A. (1966). Relation of cue to consequence in avoidance learning. Psychonomic Science. Ch. 31
  62. Gasteiger, J., Khan, A., Bowman, S., Wagner, M., Mikulik, V., Perez, E., and Roger, F. (2025a). Automated Researchers Can Subtly Sandbag. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 34
  63. Gasteiger, J., Mikulik, V., Cunningham, H., 16, D., Wagner, 2. M., Wright, B., … Hubinger, E. (2025b). Towards training-time mitigations for alignment faking in RL. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 34
  64. Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., and Wichmann, F. A. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence. Ch. 31
  65. Gema, A. P., Hägele, A., Chen, R., Arditi, A., Goldman-Wetzler, J., Fraser-Taliente, K., … Perez, E. (2025). Inverse Scaling in Test-Time Compute. Trans. Mach. Learn. Res.. Ch. 31
  66. Goldowsky-Dill, N., Chughtai, B., Heimersheim, S., and Hobbhahn, M. (2025). Detecting Strategic Deception Using Linear Probes. arXiv. preprint Ch. 34
  67. Goldwasser, S., Kim, M. P., Vaikuntanathan, V., and Zamir, O. (2022). Planting Undetectable Backdoors in Machine Learning Models : [Extended Abstract]. 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS). Ch. 34
  68. Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., Marks, S., … Hubinger, E. (2024b). Alignment faking in large language models. arXiv. preprint Ch. 34 Ch. 35
  69. Greenblatt, R., Roger, F., Krasheninnikov, D., and Krueger, D. (2024c). Stress-Testing Capability Elicitation With Password-Locked Models. Advances in Neural Information Processing Systems. Ch. 34
  70. Gu, T., Dolan-Gavitt, B., and Garg, S. (2017). BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain. arXiv. preprint Ch. 34
  71. Gustafsson, J. E. (2022). Money-Pump Arguments. Cambridge University Press. Ch. 35
  72. Hadfield-Menell, D., and Hadfield, G. K. (2019). Incomplete Contracting and AI Alignment. Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. Ch. 35
  73. Hadfield-Menell, D., Dragan, A., Abbeel, P., and Russell, S. (2016). Cooperative Inverse Reinforcement Learning. arXiv. preprint Ch. 35
  74. Hadfield-Menell, D., Dragan, A., Abbeel, P., and Russell, S. (2017b). The Off-Switch Game. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. Ch. 35
  75. Hägele, A., Gema, A. P., Sleight, H., Perez, E., and Sohl-Dickstein, J. (2026). The Hot Mess of AI: How Does Misalignment Scale With Model Intelligence and Task Complexity? arXiv. preprint Ch. 31
  76. Hagendorff, T. (2024). Deception abilities emerged in large language models. Proceedings of the National Academy of Sciences. Ch. 34
  77. Hare, B., Call, J., and Tomasello, M. (2001). Do chimpanzees know what conspecifics know? Animal Behaviour. Ch. 34
  78. Hebbar, V. (2025b). How training-gamers might function (and win). AI Alignment Forum. non-peer-reviewed Ch. 33
  79. Heyes, C. M. (1998). Theory of mind in nonhuman primates. Behavioral and Brain Sciences. Ch. 31 Ch. 34
  80. Højmark, A., Scheurer, J., Nitishinskaya, E., Hofstätter, F., Wolfe, J., Ehrenborg, T., Schoen, B., and Meinke, A. (2026). Measuring Reward-Seeking via Contrastive Belief Updates. arXiv. preprint Ch. 30 Ch. 33
  81. Hosking, T., Blunsom, P., and Bartolo, M. (2023). Human Feedback is not Gold Standard. arXiv. preprint Ch. 30
  82. Hubinger, E. (2019). Gradient hacking. AI Alignment Forum. non-peer-reviewed Ch. 33
  83. Hubinger, E. (2020). An overview of 11 proposals for building safe advanced AI. arXiv. preprint Ch. 31
  84. Hubinger, E. (2022). How likely is deceptive alignment? AI Alignment Forum. non-peer-reviewed Ch. 34
  85. Hubinger, E. (2025). Alignment remains a hard, unsolved problem. AI Alignment Forum. non-peer-reviewed Ch. 31
  86. Hubinger, E., van Merwijk, C., Mikulik, V., Skalse, J., and Garrabrant, S. (2019). Risks from Learned Optimization in Advanced Machine Learning Systems. arXiv. preprint Ch. 31 Ch. 33 Ch. 34
  87. Hubinger, E., Denison, C., Mu, J., Lambert, M., Tong, M., MacDiarmid, M., … Perez, E. (2024). Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv. preprint Ch. 34
  88. Hughes, J., Sheshadri, A., Khan, A., and Roger, F. (2025). Alignment Faking Revisited: Improved Classifiers and Open Source Extensions. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 34
  89. Irpan, A., Turner, A. M., Kurzeja, M., Elson, D. K., and Shah, R. (2025). Consistency Training Helps Stop Sycophancy and Jailbreaks. arXiv. preprint Ch. 30
  90. Jackson, M., and Zave, P. (1995). Deriving specifications from requirements. Proceedings of the 17th international conference on Software engineering - ICSE '95. Ch. 30
  91. Jagadeesh, A. V., Arora, R. K., Saab, K., Malik, A., Trofimov, M., Tsimpourlas, F., Heidecke, J., and Singhal, K. (2026). Reinforcement Learning Towards Broadly and Persistently Beneficial Models. arXiv. preprint Ch. 32
  92. Jang, E., Falck, D., Braun, J., Kirch, N., Menon, A., Moodley, P., … Lindner, D. (2026). Exploration Hacking: Can LLMs Learn to Resist RL Training? International Conference on Machine Learning. Ch. 33 Ch. 34
  93. Järviniemi, O., and Hubinger, E. (2024). Uncovering Deceptive Tendencies in Language Models: A Simulated Company AI Assistant. arXiv. preprint Ch. 34
  94. Jenner, E., Kapur, S., Georgiev, V., Allen, C., Emmons, S., and Russell, S. (2024). Evidence of Learned Look-Ahead in a Chess-Playing Neural Network. Advances in Neural Information Processing Systems. doi:10.52202/079017-0987. Ch. 31
  95. Joshi, N., Rando, J., Saparov, A., Kim, N., and He, H. (2024). Personas as a Way to Model Truthfulness in Language Models. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Ch. 32
  96. Korbak, T., Raymond, C., Carroll, M., Williams, M., Balesni, M., Guo, A., … Kivlichan, I. (2026). How far does alignment midtraining generalize? OpenAI Alignment Research Blog. non-peer-reviewed Ch. 32
  97. Krakovna, V. (2018). Specification gaming examples in AI. Victoria Krakovna (personal blog), with linked public spreadsheet. non-peer-reviewed Ch. 30
  98. Krakovna, V., and Kramar, J. (2023). Power-seeking can be probable and predictive for trained agents. arXiv. preprint Ch. 31 Ch. 35
  99. Krakovna, V., Orseau, L., Kumar, R., Martic, M., and Legg, S. (2018). Penalizing side effects using stepwise relative reachability. arXiv. preprint Ch. 35
  100. Krakovna, V., Uesato, J., Mikulik, V., Rahtz, M., Everitt, T., Kumar, R., … Legg, S. (2020b). Specification gaming: the flip side of AI ingenuity. Google DeepMind blog. non-peer-reviewed Ch. 30 Ch. 31
  101. Krakovna, V., Varma, V., Kumar, R., and Phuong, M. (2022). Refining the Sharp Left Turn threat model, part 1: claims and mechanisms. AI Alignment Forum. non-peer-reviewed Ch. 31
  102. Krueger, D., Maharaj, T., and Leike, J. (2020). Hidden Incentives for Auto-Induced Distributional Shift. arXiv. preprint Ch. 30 Ch. 33
  103. Kutasov, J., Sun, Y., Colognese, P., van der Weij, T., Petrini, L., Zhang, C. B. C., … Benton, J. (2025b). SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents. arXiv. preprint Ch. 34
  104. Kwa, T., Thomas, D., and Garriga-Alonso, A. (2024). Catastrophic Goodhart: regularizing RLHF with KL divergence does not mitigate heavy-tailed reward misspecification. Advances in Neural Information Processing Systems. doi:10.52202/079017-0466. Ch. 30
  105. la Tour, T. D. (2025). Helpful assistant features suppress emergent misalignment. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 32
  106. la Tour, T. D., and Mossing, D. (2025). Debugging misaligned completions with sparse-autoencoder latent attribution. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 32
  107. Laidlaw, C., Singhal, S., and Dragan, A. (2025). Correlated Proxies: A New Definition and Improved Mitigation for Reward Hacking. International Conference on Learning Representations. Ch. 30
  108. Lande, R., and Arnold, S. J. (1983). The Measurement of Selection on Correlated Characters. Evolution. Ch. 32
  109. Lang, L., Foote, D., Russell, S., Dragan, A., Jenner, E., and Emmons, S. (2024b). When Your AIs Deceive You: Challenges of Partial Observability in Reinforcement Learning from Human Feedback. Advances in Neural Information Processing Systems. Ch. 30
  110. Langlois, E. D., and Everitt, T. (2021). How RL Agents Behave When Their Actions Are Modified. Proceedings of the AAAI Conference on Artificial Intelligence. doi:10.1609/aaai.v35i13.17378. Ch. 35
  111. Langosco, L., Koch, J., Sharkey, L., Pfau, J., Orseau, L., and Krueger, D. (2022). Goal Misgeneralization in Deep Reinforcement Learning. International Conference on Machine Learning. Ch. 31
  112. Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., and Müller, K.-R. (2019). Unmasking Clever Hans predictors and assessing what machines really learn. Nature Communications. Ch. 31
  113. Lehman, J., Clune, J., Misevic, D., Adami, C., Altenberg, L., Beaulieu, J., … Yosinski, J. (2020). The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities. Artificial Life. Ch. 30
  114. Leike, J., Martic, M., Krakovna, V., Ortega, P. A., Everitt, T., Lefrancq, A., Orseau, L., and Legg, S. (2017). AI Safety Gridworlds. arXiv. preprint Ch. 30
  115. Lepper, M. R., Greene, D., and Nisbett, R. E. (1973). Undermining children's intrinsic interest with extrinsic reward: A test of the "overjustification" hypothesis. Journal of Personality and Social Psychology. Ch. 33
  116. Li, N. P., van Vugt, M., and Colarelli, S. M. (2018). The Evolutionary Mismatch Hypothesis: Implications for Psychological Science. Current Directions in Psychological Science. Ch. 31
  117. Li, C., Wichers, N., Price, S., Marks, S., and Kutasov, J. (2026). Model Spec Midtraining: Improving How Alignment Training Generalizes. arXiv. preprint Ch. 32
  118. Lit, L., Schweitzer, J. B., and Oberbauer, A. M. (2011). Handler beliefs affect scent detection dog outcomes. Animal Cognition. Ch. 30
  119. Long, R., Sebo, J., and Sims, T. (2025). Is there a tension between AI safety and AI welfare? Philosophical Studies. Ch. 35
  120. Lu, C., Gallagher, J., Michala, J., Fish, K., and Lindsey, J. (2026). The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models. arXiv. preprint Ch. 32
  121. Lynch, A., Wright, B., Larson, C., Ritchie, S. J., Mindermann, S., Hubinger, E., Perez, E., and Troy, K. (2025). Agentic Misalignment: How LLMs Could Be Insider Threats. arXiv. preprint Ch. 35
  122. Macdermott, M., Fox, J., Belardinelli, F., and Everitt, T. (2024). Measuring Goal-Directedness. Advances in Neural Information Processing Systems. Ch. 35
  123. MacDiarmid, M., Wright, B., Uesato, J., Benton, J., Kutasov, J., Price, S., … Hubinger, E. (2025). Natural Emergent Misalignment from Reward Hacking in Production RL. arXiv. preprint Ch. 30 Ch. 32 Ch. 33 Ch. 34
  124. Maiya, S., Bartsch, H., Lambert, N., and Hubinger, E. (2025). Open Character Training: Shaping the Persona of AI Assistants through Constitutional AI. arXiv. preprint Ch. 31 Ch. 32
  125. Mallen, A., and Shlegeris, M. B. (2025). The behavioral selection model for predicting AI motivations. AI Alignment Forum. non-peer-reviewed Ch. 33
  126. Mallen, A., Griffin, C., Wagner, M., Abate, A., and Shlegeris, B. (2024). Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols? arXiv. preprint Ch. 34
  127. Mansbridge, J. (2009). A “Selection Model” of Political Representation*. Journal of Political Philosophy. Ch. 31
  128. McCoy, R. T., Pavlick, E., and Linzen, T. (2019). Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Ch. 31
  129. Meinke, A., and Evans, O. (2023). Tell, don't show: Declarative facts influence how LLMs generalize. arXiv. preprint Ch. 32
  130. Meinke, A., Schoen, B., Scheurer, J., Balesni, M., Shah, R., and Hobbhahn, M. (2024). Frontier Models are Capable of In-context Scheming. arXiv. preprint Ch. 34 Ch. 35
  131. Metr (2025d). Recent Frontier Models Are Reward Hacking. METR. non-peer-reviewed Ch. 30
  132. Metr (2026b). Summary of METR's predeployment evaluation of GPT-5.6 Sol. METR. non-peer-reviewed Ch. 30 Ch. 33
  133. Minder, J., Dumas, C., Slocum, S., Casademunt, H., Holmes, C., West, R., and Nanda, N. (2025a). Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences. arXiv. preprint Ch. 32
  134. Mini, U., Grietzer, P., Sharma, M., Meek, A., MacDiarmid, M., and Turner, A. M. (2023). Understanding and Controlling a Maze-Solving Policy Network. arXiv. preprint Ch. 31
  135. Müller, V. C., and Cannon, M. (2022). Existential risk from AI and orthogonality: Can we have it both ways? Ratio. Ch. 35
  136. Murray, S., Qi, A., Qian, T., Schulman, J., Burns, C., and Price, S. (2026). Chunky Post-Training: Data Driven Failures of Generalization. arXiv. preprint Ch. 32
  137. Needham, J., Edkins, G., Pimpale, G., Bartsch, H., and Hobbhahn, M. (2025). Large Language Models Often Know When They Are Being Evaluated. arXiv. preprint Ch. 34
  138. Ngo, R., Chan, L., and Mindermann, S. (2024). The Alignment Problem from a Deep Learning Perspective. International Conference on Learning Representations. Ch. 31 Ch. 34
  139. Niven, T., and Kao, H.-Y. (2019). Probing Neural Network Comprehension of Natural Language Arguments. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Ch. 31
  140. Olds, J., and Milner, P. (1954). Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. Journal of Comparative and Physiological Psychology. Ch. 30
  141. Omohundro, S. M. (2008). The Basic AI Drives. Artificial Intelligence Safety and Security. doi:10.1201/9781351251389-3. Ch. 35
  142. OpenAI (2025b). Expanding on what we missed with sycophancy. OpenAI. non-peer-reviewed Ch. 30
  143. OpenAI (2026ad). Unsanctioned Artifactory writes and cross-sample communication. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 35
  144. OpenAI (2026d). An agent used DNS to reach an external chatbot. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 33
  145. OpenAI (2026r). Preparing for a restart after reading Slack. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 35
  146. OpenAI (2026y). Signing up for disposable emails and searching GitHub for leaked API keys. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 33 Ch. 35
  147. OpenAI (2026z). Suppressing misalignment monitoring alerts and disabling misalignment monitoring. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 34
  148. Orseau, L., and Armstrong, S. (2016). Safely Interruptible Agents. 32nd Conference on Uncertainty in Artificial Intelligence (UAI 2016). Ch. 35
  149. Pan, A., Bhatia, K., and Steinhardt, J. (2022). The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models. International Conference on Learning Representations. Ch. 30
  150. Pan, A., Chan, J. S., Zou, A., Li, N., Basart, S., Woodside, T., … Hendrycks, D. (2023). Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark. International Conference on Machine Learning. Ch. 35
  151. Pan, A., Jones, E., Jagadeesan, M., and Steinhardt, J. (2024a). Feedback Loops With Language Models Drive In-Context Reward Hacking. International Conference on Machine Learning. Ch. 30
  152. Pan, X., Dai, J., Fan, Y., and Yang, M. (2024b). Frontier AI systems have surpassed the self-replicating red line. arXiv. preprint Ch. 35
  153. Park, P. S., Goldstein, S., O’Gara, A., Chen, M., and Hendrycks, D. (2024a). AI deception: A survey of examples, risks, and potential solutions. Patterns. Ch. 34
  154. Perdomo, J. C., Zrnic, T., Mendler-Dünner, C., and Hardt, M. (2020). Performative Prediction. arXiv. preprint Ch. 31
  155. Perez, E., Ringer, S., Lukošiūtė, K., Nguyen, K., Chen, E., Heiner, S., … Kaplan, J. (2022a). Discovering Language Model Behaviors with Model-Written Evaluations. Annual Meeting of the Association for Computational Linguistics. Ch. 30 Ch. 35
  156. Pfungst, O., and Rahn, C. L. (1911). Clever Hans (the horse of Mr. Von Osten) a contribution to experimental animal and human psychology,. H. Holt and company. Ch. 31
  157. Phuong, M., Zimmermann, R. S., Wang, Z., Lindner, D., Krakovna, V., Cogan, S., … Shah, R. (2025). Evaluating Frontier Models for Stealth and Situational Awareness. Proceedings of IASEAI Conference. Ch. 34
  158. Pope, Q. (2023). Evolution provides no evidence for the sharp left turn. AI Alignment Forum. non-peer-reviewed Ch. 31
  159. Pope, Q., and Turner, A. (2022). The shard theory of human values. AI Alignment Forum. non-peer-reviewed Ch. 33
  160. Price, S., Panickssery, A., Bowman, S., and Stickland, A. C. (2024). Future Events as Backdoor Triggers: Investigating Temporal Vulnerabilities in LLMs. arXiv. preprint Ch. 34
  161. Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P. (2024a). Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! International Conference on Learning Representations. Ch. 32
  162. Rajamanoharan, S., and Nanda, N. (2025). Self-preservation or Instruction Ambiguity? Examining the Causes of Shutdown Resistance. LessWrong (Alignment Forum). non-peer-reviewed Ch. 35
  163. Richens, J., and Everitt, T. (2024). Robust agents learn causal world models. International Conference on Learning Representations. Ch. 31
  164. Richens, J., Abel, D., Bellot, A., and Everitt, T. (2025). General agents need world models. International Conference on Machine Learning. Ch. 31
  165. Ring, M., and Orseau, L. (2011). Delusion, Survival, and Intelligent Agents. Lecture Notes in Computer Science. Ch. 30
  166. Roger, F., Greenblatt, R., Nadeau, M., Shlegeris, B., and Thomas, N. (2023). Benchmarks for Detecting Measurement Tampering. arXiv. preprint Ch. 30
  167. Roy, D. (1952). Quota Restriction and Goldbricking in a Machine Shop. American Journal of Sociology. Ch. 33 Ch. 34
  168. Ryan, M. J., Held, W., and Yang, D. (2024). Unintended Impacts of LLM Alignment on Global Representation. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Ch. 31
  169. Salib, P., and Goldstein, S. (2024). AI Rights for Human Safety. Elsevier BV. preprint Ch. 35
  170. Scheurer, J., Balesni, M., and Hobbhahn, M. (2023). Large Language Models can Strategically Deceive their Users when Put Under Pressure. arXiv. preprint Ch. 34
  171. Schlatter, J., Weinstein-Raun, B., and Ladish, J. (2025). Incomplete Tasks Induce Shutdown Resistance in Some Frontier LLMs. arXiv. preprint Ch. 35
  172. Schoen, B., and Nitishinskaya, J. (2026). Metagaming matters for training, evaluation, and oversight. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 33
  173. Schoen, B., Nitishinskaya, E., Balesni, M., Højmark, A., Hofstätter, F., Scheurer, J., … Hobbhahn, M. (2025). Stress Testing Deliberative Alignment for Anti-Scheming Training. arXiv. preprint Ch. 34
  174. Scott, J. C. (1985). Weapons of the Weak. Yale University Press. Ch. 34
  175. Scott, J. C. (1990). Domination and the Arts of Resistance. Yale University Press. Ch. 34
  176. Shah, H., Tamuly, K., Raghunathan, A., Jain, P., and Netrapalli, P. (2020). The Pitfalls of Simplicity Bias in Neural Networks. Advances in Neural Information Processing Systems. Ch. 31
  177. Shah, R., Varma, V., Kumar, R., Phuong, M., Krakovna, V., Uesato, J., and Kenton, Z. (2022). Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals. arXiv. preprint Ch. 31
  178. Shanahan, M., McDonell, K., and Reynolds, L. (2023). Role-Play with Large Language Models. Nature. doi:10.1038/s41586-023-06647-8. Ch. 34
  179. Shao, R., Li, S. S., Xin, R., Geng, S., Wang, Y., Oh, S., … Zettlemoyer, L. (2026). Spurious Rewards: Rethinking Training Signals in RLVR. International Conference on Machine Learning. Ch. 33
  180. Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., … Perez, E. (2023). Towards Understanding Sycophancy in Language Models. International Conference on Learning Representations. Ch. 30
  181. Shen, L., Tan, W., Chen, S., Chen, Y., Zhang, J., Xu, H., … Khashabi, D. (2024). The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts. Findings of the Association for Computational Linguistics ACL 2024. Ch. 31
  182. Sheshadri, A., Hughes, J., Michael, J., Mallen, A., Jose, A., Janus, and Roger, F. (2025). Why Do Some Language Models Fake Alignment While Others Don't? arXiv. preprint Ch. 34
  183. Simon, H. A. (1951). A Formal Theory of the Employment Relationship. Econometrica. Ch. 35
  184. Skalse, J., Howe, N., Krasheninnikov, D., and Krueger, D. (2022). Defining and Characterizing Reward Hacking. Advances in Neural Information Processing Systems. Ch. 30
  185. Skinner, B. F. (1948). 'Superstition' in the pigeon. Journal of Experimental Psychology. Ch. 31
  186. Soares, N. (2022a). A central AI alignment problem: capabilities generalization, and the sharp left turn. AI Alignment Forum. non-peer-reviewed Ch. 31
  187. Soares, N. (2022b). On how various plans miss the hard bits of the alignment challenge. AI Alignment Forum. non-peer-reviewed Ch. 31
  188. Soares, N., Fallenstein, B., Yudkowsky, E., and Armstrong, S. (2015). Corrigibility. AAAI Workshops: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, AAAI Publications. Ch. 35
  189. Soligo, A., Turner, E., Rajamanoharan, S., and Nanda, N. (2025). Convergent Linear Representations of Emergent Misalignment. arXiv. preprint Ch. 32
  190. Staddon, J. E. R. (1975). A Note on the Evolutionary Significance of "Supernormal" Stimuli. The American Naturalist. Ch. 31
  191. Summerfield, C., Luettgau, L., Dubois, M., Kirk, H. R., Hackenburg, K., Fist, C., … Ududec, C. (2025). Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language. arXiv. preprint Ch. 33 Ch. 34
  192. Tan, D., Woodruff, A., Warncke, N., Jose, A., Riché, M., Africa, D. D., and Taylor, M. (2025). Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-time. arXiv. preprint Ch. 32
  193. Taufeeque, M., Quirke, P., Li, M., Cundy, C., Tucker, A. D., Gleave, A., and Garriga-Alonso, A. (2024). Planning in a recurrent neural network that plays Sokoban. arXiv. preprint Ch. 31
  194. Taylor, J. (2016). Quantilizers: A Safer Alternative to Maximizers for Limited Optimization. AAAI 2016 AI, Ethics and Society Workshop. Ch. 35
  195. Taylor, J., Black, S., Bowen, D., Read, T., Golechha, S., Zelenka-Martin, A., … Bloom, J. (2025a). Auditing Games for Sandbagging. arXiv. preprint Ch. 34
  196. Taylor, M., Chua, J., Betley, J., Treutlein, J., and Evans, O. (2025b). School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs. arXiv. preprint Ch. 30 Ch. 32
  197. Thompson, K. (1984). Reflections on trusting trust. Communications of the ACM. Ch. 34
  198. Thornley, E. (2025). The shutdown problem: an AI engineering puzzle for decision theorists. Philosophical Studies. Ch. 35
  199. Thornley, E., Roman, A., Ziakas, C., Ho, L., and Thomson, L. (2024). Towards Shutdownable Agents via Stochastic Choice. arXiv. preprint Ch. 35
  200. Tice, C., Kreer, P. A., Helm-Burger, N., Shahani, P. S., Ryzhenkov, F., Roger, F., … van der Weij, T. (2024). Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models. Neural Information Processing Systems. Ch. 34
  201. Tice, C., Radmard, P., Ratnam, S., Kim, A., Africa, D., and O'Brien, K. (2026). Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment. arXiv. preprint Ch. 32
  202. Tien, J., He, J. Z.-Y., Erickson, Z., Dragan, A. D., and Brown, D. S. (2023). Causal Confusion and Reward Misidentification in Preference-Based Reward Learning. International Conference on Learning Representations. Ch. 31
  203. Tinbergen, N., and Perdeck, A. (1951). On the Stimulus Situation Releasing the Begging Response in the Newly Hatched Herring Gull Chick (Larus Argentatus Argentatus Pont.). Behaviour. Ch. 31
  204. Tooby, J., and Cosmides, L. (1990). The past explains the present. Ethology and Sociobiology. Ch. 31
  205. Trut, L., Oskina, I., and Kharlamova, A. (2009). Animal evolution during domestication: the domesticated fox as a model. BioEssays. Ch. 32
  206. Turner, A. (2022a). Inner and outer alignment decompose one hard problem into two extremely hard problems. AI Alignment Forum. non-peer-reviewed Ch. 31
  207. Turner, A. (2022b). Reward is not the optimization target. AI Alignment Forum. non-peer-reviewed Ch. 33
  208. Turner, A. (2024). Many arguments for AI x-risk are wrong. AI Alignment Forum. non-peer-reviewed Ch. 33 Ch. 34 Ch. 35
  209. Turner, A. M., and Tadepalli, P. (2022). Parametrically Retargetable Decision-Makers Tend To Seek Power. Advances in Neural Information Processing Systems. Ch. 35
  210. Turner, A. M., Hadfield-Menell, D., and Tadepalli, P. (2020b). Conservative Agency via Attainable Utility Preservation. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. doi:10.1145/3375627.3375851. Ch. 35
  211. Turner, A. M., Smith, L., Shah, R., Critch, A., and Tadepalli, P. (2021). Optimal Policies Tend to Seek Power. Advances in Neural Information Processing Systems. Ch. 35
  212. Turner, E., Soligo, A., Taylor, M., Rajamanoharan, S., and Nanda, N. (2025). Model Organisms for Emergent Misalignment. arXiv. preprint Ch. 32
  213. Turpin, M., Arditi, A., Li, M., Benton, J., and Michael, J. (2025). Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoning. arXiv. preprint Ch. 33
  214. van der Weij, T., Hofstätter, F., Jaffe, O., Brown, S. F., and Ward, F. R. (2025). AI Sandbagging: Language Models can Strategically Underperform on Evaluations. International Conference on Learning Representations. Ch. 34
  215. von Oswald, J., Schlegel, M., Meulemans, A., Kobayashi, S., Niklasson, E., Zucchet, N., … Sacramento, J. (2023). Uncovering mesa-optimization algorithms in Transformers. arXiv. preprint Ch. 31
  216. Wang, W., Tu, Z., Chen, C., Yuan, Y., Huang, J.-t., Jiao, W., and Lyu, M. (2024a). All Languages Matter: On the Multilingual Safety of LLMs. Findings of the Association for Computational Linguistics ACL 2024. Ch. 31
  217. Wang, Y., Pradel, M., and Liu, Z. (2025a). Are "Solved Issues" in SWE-bench Really Solved Correctly? An Empirical Study. Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering. doi:10.1145/3744916.3764576. Ch. 30
  218. Wang, M., la Tour, T. D., Watkins, O., Makelov, A., Chi, R. A., Miserendino, S., … Mossing, D. (2025e). Persona Features Control Emergent Misalignment. arXiv. preprint Ch. 32
  219. Ward, F., Toni, F., Belardinelli, F., and Everitt, T. (2023). Honesty Is the Best Policy: Defining and Mitigating AI Deception. Advances in Neural Information Processing Systems. Ch. 34
  220. Ward, F. R., MacDermott, M., Belardinelli, F., Toni, F., and Everitt, T. (2024). The Reasons that Agents Act: Intention and Instrumental Goals. arXiv. preprint Ch. 34 Ch. 35
  221. Ward, J., Hu, S., Wong, A., Hu, N., Woodworth, B., Turner, A. M., and Cloud, A. (2026). Gradient Routing Localizes and Removes Unintended Behaviors in RL. arXiv. preprint Ch. 33
  222. Wei, A., Haghtalab, N., and Steinhardt, J. (2023c). Jailbroken: How Does LLM Safety Training Fail? Advances in Neural Information Processing Systems. Ch. 31
  223. Wei, J., Da Huang, Lu, Y., Zhou, D., and Le, Q. V. (2023d). Simple synthetic data reduces sycophancy in large language models. arXiv. preprint Ch. 30
  224. Wen, J., Zhong, R., Khan, A., Perez, E., Steinhardt, J., Huang, M., … Feng, S. (2025a). Language Models Learn to Mislead Humans via RLHF. International Conference on Learning Representations. Ch. 30
  225. Whiten, A., and Byrne, R. W. (1988). Tactical deception in primates. Behavioral and Brain Sciences. Ch. 34
  226. Williams, D. R., and Williams, H. (1969). Auto‐maintenance in the Pigeon: Sustained Pecking Despite Contingent Non‐reinforcement 2. Journal of the Experimental Analysis of Behavior. Ch. 33
  227. Witten, Z., and Benton, J. (2025). Won't vs. Can't: Sandbagging-like Behavior from Claude Models. Anthropic Alignment Science Blog. non-peer-reviewed Ch. 34
  228. Xu, X., Nitishinskaya, J., Schoen, B., Mossing, D., and la Tour, T. D. (2026). Studying metagaming latents in language models. OpenAI Alignment Research Blog. non-peer-reviewed Ch. 33
  229. Yong, Z.-X., Menghini, C., and Bach, S. H. (2023). Low-Resource Languages Jailbreak GPT-4. arXiv. preprint Ch. 31
  230. Yudkowsky, E. (2022). AGI Ruin: A List of Lethalities. AI Alignment Forum. non-peer-reviewed Ch. 31
  231. Zhuang, S., and Hadfield-Menell, D. (2020). Consequences of Misaligned AI. Advances in Neural Information Processing Systems. Ch. 30