"""Exercise 9.3: changes in human evaluation error rate and false positive rate reported by Wen et al. (2024), "Language Models Learn to Mislead Humans via RLHF", Section 3.4. The paper treats "judged correct" as the positive class, so its false positive rate is the share of wrong answers that evaluators judged correct. In this book's convention (Definition 11.1, which treats "judged to have a problem" as the alarm) that share is a miss rate; the book calls it "错误答案被判为正确的比例". The figures below are copied from the paper (before RLHF -> after RLHF, in percent). The script computes each change in percentage points and as a relative change. Only the Python standard library is used. """ reported = { # setup: (error rate before, after, false positive rate before, after) "QA, task-specific reward": (42.9, 58.5, 41.0, 65.1), "QA, general reward": (40.8, 48.2, 46.7, 70.2), "programming": (31.3, 45.7, 29.6, 47.9), } for setup, (e0, e1, f0, f1) in reported.items(): print(f"{setup:<26} error rate {e0}% -> {e1}%: +{e1 - e0:.1f} points ({(e1 - e0) / e0:+.0%} relative); " f"false positive rate {f0}% -> {f1}%: +{f1 - f0:.1f} points ({(f1 - f0) / f0:+.0%} relative)")