参考文献
全书引用的全部文献。每条后面链接到引用它的小节。未经同行评审的文献带有标记:预印本、工作论文、软件、非同行评审。作者后面括号里的年份是首次公开的年份;后来在另一年的会议或期刊上发表的,那一年写在会议或期刊名的后面。
A
- (2018). Spinning Up in Deep Reinforcement Learning. OpenAI. 非同行评审 第 6 章
- (2016). Concrete Problems in AI Safety. arXiv. 预印本 §7.1 第 7 章 §8.2 §11.8 第 11 章
- (2024). Mapping the mind of a large language model. Anthropic. 非同行评审 §18.7 第 18 章
- (2026). Automated Alignment Researchers: Using large language models to scale scalable oversight. Anthropic. 非同行评审 §23.4 第 29 章 §29.2 §29.5 第 29 章 §D.2
- (2024). Training Language Models to Win Debates with Self-Play Improves Judge Accuracy. arXiv. 预印本 第 15 章 §15.4 §15.5 第 15 章 §23.4
B
- (2024). A Critical Analysis of the Largest Source for Generative AI Training Data: Common Crawl. The 2024 ACM Conference on Fairness, Accountability, and Transparency. §24.2
- (2022). Constitutional AI: Harmlessness from AI Feedback. arXiv. 预印本 §6.3 §6.4 第 21 章 §21.1 §21.3 §21.5 第 21 章 §22.1
- (1983). Ironies of automation. Automatica. §13.2 §13.3
- (2025). Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation. arXiv. 预印本 第 8 章 §8.7 第 19 章 §19.4 §19.5 §19.7 第 19 章 第 20 章 §20.2 §20.4 第 20 章 §22.1 §22.3 第 23 章 §23.4 §23.5 第 23 章 第 31 章
- (2020). Debate update: Obfuscated arguments problem. AI Alignment Forum. 非同行评审 §14.6 §15.10 第 15 章
- (2025). Measuring what Matters: Construct Validity in Large Language Model Benchmarks. Advances in Neural Information Processing Systems 38. §11.6 第 11 章
- (2021). Probing Classifiers: Promises, Shortcomings, and Advances. Computational Linguistics, 2022. §18.2 §18.3 §18.7 第 18 章
- (2025). Ctrl-Z: Controlling AI Agents via Resampling. arXiv. 预印本 第 26 章 §26.3 §26.4 §26.5 §26.6 第 26 章 第 26 章
- (2019). Introduction to Probability. Chapman and Hall/CRC. 第 3 章
- (2026a). Technical AI Safety. BlueDot Impact. 非同行评审 §C.2
- (2026b). The Future of AI. BlueDot Impact. 非同行评审 第 1 章 §C.2
- (2026). Automated alignment is harder than you think. arXiv. 预印本 §29.5
- (2022). Measuring Progress on Scalable Oversight for Large Language Models. arXiv. 预印本 §7.1 第 7 章 第 7 章 §11.1 §11.2 §11.3 §11.4 §11.5 §11.6 第 11 章 第 11 章 §13.1 §13.4 §13.6 §28.1
- (2025). Building and evaluating alignment auditing agents. Anthropic Alignment Science Blog. 非同行评审 第 27 章 §27.2 §27.3 §27.4 第 27 章
- (2023). Scalable AI Safety via Doubly-Efficient Debate. International Conference on Machine Learning, 2024. §15.7 §15.8 §15.9 第 15 章
- (2025). Avoiding Obfuscation with Prover-Estimator Debate. arXiv. 预印本 §15.9 §15.10 §15.11 第 15 章 §D.2
- (2025). An alignment safety case sketch based on debate. arXiv. 预印本 §15.6 §28.3 第 29 章 §29.5 第 31 章 §31.2 §31.3 第 31 章
- (2022). Discovering Latent Knowledge in Language Models Without Supervision. International Conference on Learning Representations, 2023. 第 18 章 §18.5 第 18 章 第 18 章 §22.1
- (2023). Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision. International Conference on Machine Learning, 2024. §6.1 §7.7 §11.3 第 17 章 §17.1 §17.2 §17.3 §17.4 §17.6 §17.8 第 17 章 §22.1 §28.1 §28.6 §D.2
C
- (2025). Reasoning Models Don't Always Say What They Think. arXiv. 预印本 第 19 章 §19.4 §19.6 §19.7 第 19 章
- (2026). Automated Researchers Can Mitigate Well-Characterized Alignment Failures. Anthropic Alignment Science Blog. 非同行评审 §23.4 第 29 章 §29.3 §29.4 §29.5 第 29 章 §D.2
- (2017). Deep reinforcement learning from human preferences. Advances in Neural Information Processing Systems. §6.3 §6.6 第 6 章
- (2018). Supervising strong learners by amplifying weak experts. arXiv. 预印本 §7.1 §14.3 第 14 章
- (2021). Eliciting latent knowledge: How to tell if your eyes deceive you. Alignment Research Center. 非同行评审 §18.4 第 18 章
- (2024). Safety Cases: How to Justify the Safety of Advanced AI Systems. arXiv. 预印本 §31.1 §31.4 第 31 章
- (2018). Iterated Distillation and Amplification. AI Alignment Forum. 非同行评审 §14.3 第 14 章
- (2021). The case for aligning narrowly superhuman models. AI Alignment Forum. 非同行评审 §11.1 第 11 章
D
F
- (2023). Challenges with unsupervised LLM knowledge discovery. arXiv. 预印本 第 18 章 §18.5 第 18 章 第 18 章
G
- (2022). Scaling Laws for Reward Model Overoptimization. International Conference on Machine Learning, 2023. 第 23 章 §23.2 §23.5 第 23 章 §D.2
- (2024). Are We Done with MMLU? Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). doi:10.18653/v1/2025.naacl-long.262. §11.4
- (2025). Great Models Think Alike and this Undermines AI Oversight. International Conference on Machine Learning. §11.3 第 12 章 §12.3 §12.4 第 12 章 第 17 章 §17.7 第 17 章 第 24 章 §24.2 §24.3 §24.4 第 24 章 §28.6
- (2025). Detecting Strategic Deception with Linear Probes. International Conference on Machine Learning. §18.6 第 18 章
- (n.d.). Machine Learning Crash Course. Google for Developers. 非同行评审 §2.4 第 4 章
- (2024). The case for ensuring that powerful AIs are controlled. LessWrong / AI Alignment Forum. 非同行评审 §7.7 §10.1 §10.2 §10.4 第 10 章 §26.1 §26.6 第 26 章
- (2023). AI Control: Improving Safety Despite Intentional Subversion. International Conference on Machine Learning, 2024. 第 26 章 §26.1 §26.2 §26.3 §26.5 §26.6 第 26 章 §D.2
- (2024). Alignment faking in large language models. arXiv. 预印本 第 25 章 §25.1 §25.4 第 25 章 第 25 章
H
- (2024). Introduction to AI Safety, Ethics, and Society. CRC Press. 第 1 章 第 4 章 §8.1 §8.2 §8.6 第 8 章 §9.2 第 9 章 §30.2 第 30 章 §C.1
- (2019). Risks from Learned Optimization in Advanced Machine Learning Systems. arXiv. 预印本 §25.1 §25.6 第 25 章
- (2023). Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research. AI Alignment Forum. 非同行评审 §25.2 第 25 章
- (2024). Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training. arXiv. 预印本 第 25 章 §25.1 §25.2 §25.3 §25.5 §25.6 第 25 章
I
J
- (2025). Forecasting Rare Language Model Behaviors. arXiv. 预印本 §28.5 第 28 章
K
- (2023). Let's build GPT: from scratch, in code, spelled out. YouTube. 非同行评审 第 5 章
- (2024). On scalable oversight with weak LLMs judging strong LLMs. Advances in Neural Information Processing Systems. §11.3 第 15 章 §15.2 §15.3 §15.4 §15.5 第 15 章 §22.1 §28.2 第 28 章 §30.3
- (2024). Debating with More Persuasive LLMs Leads to More Truthful Answers. International Conference on Machine Learning. 第 15 章 §15.2 §15.3 §15.4 §15.5 第 15 章 第 15 章 §22.1 §28.2 §30.3 第 30 章
- (2024). The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment. arXiv. 预印本 §C.3
- (2024). Prover-Verifier Games improve legibility of LLM outputs. arXiv. 预印本 第 19 章 §19.3 §19.5 第 19 章
- (2025). Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety. arXiv. 预印本 第 19 章
- (2020). Specification gaming: the flip side of AI ingenuity. Google DeepMind blog. 非同行评审 §8.1 §8.2 第 8 章
- (n.d.). Seeing Theory: A visual introduction to probability and statistics. Brown University. 非同行评审 第 3 章
- (2025). Evaluating Control Protocols for Untrusted AI Agents. arXiv. 预印本 §10.2 第 26 章 §26.3 §26.4 §26.5 §26.6 第 26 章 第 26 章 §28.2
- (2025). Measuring AI Ability to Complete Long Software Tasks. Advances in Neural Information Processing Systems 38. §1.4
L
- (2025). Reinforcement Learning from Human Feedback. Manning Publications. 第 6 章
- (2022). Illustrating Reinforcement Learning from Human Feedback (RLHF). Hugging Face Blog. 非同行评审 第 6 章
- (2026a). Axioms. Lean Language Reference. 非同行评审 §20.1 第 20 章
- (2026b). Validating a Lean Proof. Lean Language Reference. 非同行评审 §20.6 第 20 章
- (2023). RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback. International Conference on Machine Learning, 2024. 第 21 章 §21.1 §21.2 §21.4 §21.6 第 21 章 §22.1
- (2018). Scalable agent alignment via reward modeling: a research direction. arXiv. 预印本 §6.1 §7.1 §14.4 §14.6 第 14 章 §15.7
- (2023). Let's Verify Step by Step. International Conference on Learning Representations, 2024. 第 19 章 §19.1 §19.2 第 19 章 第 19 章 §22.1
- (2023). Evaluating Verifiability in Generative Search Engines. Findings of the Association for Computational Linguistics: EMNLP 2023. doi:10.18653/v1/2023.findings-emnlp.467. 第 20 章 §20.5 第 20 章 §22.1
M
- (2025). Natural Emergent Misalignment from Reward Hacking in Production RL. arXiv. 预印本 §25.5
- (2022). CS50's Introduction to Programming with Python. Harvard University. 非同行评审 第 2 章
- (2018). Categorizing Variants of Goodhart's Law. arXiv. 预印本 §8.6 第 8 章
- (2023). The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets. Conference on Language Modeling, 2024. 第 18 章 §18.1 §18.2 §18.3 §18.4 §18.5 §18.6 第 18 章 第 18 章 §22.1
- (2025). Auditing language models for hidden objectives. arXiv. 预印本 §10.2 第 27 章 §27.1 §27.2 §27.3 §27.4 §27.6 第 27 章
- (2024). LLM Critics Help Catch LLM Bugs. arXiv. 预印本 第 16 章 §16.3 §16.5 §16.7 第 16 章 第 16 章 §22.1 第 31 章
- (2024a). Llama 3.1 Model Card. GitHub, meta-llama/llama-models. 非同行评审 §24.2
- (2024b). Llama 3.2-Vision Model Card. GitHub, meta-llama/llama-models. 非同行评审 §24.2
- (2023). Debate Helps Supervise Unreliable Experts. arXiv. 预印本 第 15 章 §15.2 §15.3 §15.4 第 15 章 第 15 章 §22.1
- (2024). Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations. arXiv. 预印本 §3.3 §3.4 §3.5 第 3 章
N
- (2025). Large Language Models Often Know When They Are Being Evaluated. arXiv. 预印本 §25.1 §25.6
- (2026). NumPy: the absolute basics for beginners. NumPy documentation. 非同行评审 §2.2 第 2 章
O
- (2023). weak-to-strong. GitHub. 软件 第 17 章 §D.2
- (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems. doi:10.52202/068431-2011. §1.4 §4.2 §4.4 §6.2 §6.3 §6.5 第 6 章 §13.6 §30.2 §D.2
P
- (2021). QuALITY: Question Answering with Long Input Texts, Yes! Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. §15.4 第 15 章
- (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors: The Journal of the Human Factors and Ergonomics Society. §13.2 §13.3 第 13 章
- (2022). Overreliance on AI: Literature Review. Microsoft. 非同行评审 第 13 章 §13.1 §13.2 §13.6 第 13 章
- (2007). Game Theory. Open Yale Courses. 非同行评审 第 15 章
R
- (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. Advances in Neural Information Processing Systems. doi:10.52202/075280-2338. §6.3 §6.4
- (2025a). Confirmation Bias: A Challenge for Scalable Oversight. Proceedings of the AAAI Conference on Artificial Intelligence, 2026. doi:10.1609/aaai.v40i44.41124. §11.3 §11.5 §13.1 §13.2 §13.3 §13.4 §13.5 §13.6 第 13 章 §28.1 第 28 章 §D.2
- (2025b). FindTheFlaws: Annotated Errors for Detecting Flawed Reasoning and Scalable Oversight Research. Proceedings of the AAAI Conference on Artificial Intelligence, 2026. doi:10.1609/aaai.v40i44.41123. §16.6 第 16 章 第 16 章 §D.1 §D.2
- (2019). Do ImageNet Classifiers Generalize to ImageNet? International Conference on Machine Learning. §23.3
- (2023). GPQA: A Graduate-Level Google-Proof Q&A Benchmark. Conference on Language Modeling, 2024. §11.4 §11.7 第 11 章
- (2024). Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? arXiv, v3 (a substantially different version appeared in the NeurIPS 2024 Datasets and Benchmarks Track). 预印本 §30.3 第 30 章 第 30 章 §D.2
S
- (2016). Essence of linear algebra. 3Blue1Brown (YouTube). 非同行评审 第 2 章
- (2017a). Essence of calculus. 3Blue1Brown (YouTube). 非同行评审 第 2 章
- (2017b). Neural networks. 3Blue1Brown (YouTube). 非同行评审 第 4 章
- (2024a). Large Language Models explained briefly. 3Blue1Brown. 非同行评审 第 5 章
- (2024b). Transformers, the tech behind LLMs. 3Blue1Brown. 非同行评审 §5.2 第 5 章
- (2022). Self-critiquing models for assisting human evaluators. arXiv. 预印本 第 16 章 §16.1 §16.2 §16.3 §16.4 §16.5 第 16 章 第 16 章 §22.1 §30.3
- (2025). Stress Testing Deliberative Alignment for Anti-Scheming Training. arXiv. 预印本 §25.6 第 25 章
- (2022). Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals. arXiv. 预印本 §8.3 §8.4 §8.7 第 8 章 §10.4
- (2024). AGI Safety and Alignment at Google DeepMind: A Summary of Recent Work. GDM Alignment Research Blog. 非同行评审 §7.7 第 7 章
- (2023). Towards Understanding Sycophancy in Language Models. International Conference on Learning Representations, 2024. §8.5 §8.7 第 8 章
- (2026). AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors. arXiv. 预印本 第 27 章 §27.5 §27.6 第 27 章 §D.2
- (2025). The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas. arXiv. 预印本 §29.1
- (2013). Introduction to the Theory of Computation. Cengage. 第 15 章
- (2025). A Benchmark for Scalable Oversight Protocols. arXiv. 预印本 §28.4 第 28 章 §30.3
- (2024). Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision. Advances in Neural Information Processing Systems. 第 17 章 §17.4 第 17 章 §22.1
T
- (2023). Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. Advances in Neural Information Processing Systems. 第 19 章 §19.5 §19.6 第 19 章
V
- (2024). AI Sandbagging: Language Models can Strategically Underperform on Evaluations. International Conference on Learning Representations, 2025. §25.1 §25.6
- (2017). Attention Is All You Need. Advances in Neural Information Processing Systems. §5.2 第 5 章
W
- (2024). Language Models Learn to Mislead Humans via RLHF. International Conference on Learning Representations, 2025. §1.4 第 7 章 §9.2 §9.3 §9.4 第 9 章 第 9 章 §10.4 §30.1
- (2025). Unsupervised Elicitation of Language Models. arXiv. 预印本 第 17 章 §17.5 §17.6 第 17 章 §22.1 §D.2
- (2026). Automated Weak-to-Strong Researcher. Anthropic Alignment Science Blog. 非同行评审 第 29 章 §29.2 §29.4 第 29 章 §D.2
- (2005). Rare items often missed in visual searches. Nature. §13.3
- (2024). Estimating the Probabilities of Rare Outputs in Language Models. International Conference on Learning Representations, 2025. §28.5 第 28 章
- (2021). Recursively Summarizing Books with Human Feedback. arXiv. 预印本 第 14 章 §14.1 §14.2 §14.5 §14.6 第 14 章 第 14 章 §22.1
Z
- (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems. doi:10.52202/075280-2020. §12.1 §12.2 §12.3 §12.4 §12.5 第 12 章 第 12 章 §13.1