An Unverifiable WorldPreface: A World Without OraclesContents中文

Preface: A World Without Oracles

Before sailing, marrying, or founding a city, the ancient Greeks would go to Delphi and ask the oracle. The oracle mattered not because it was always accurate, but because it promised one thing: before you acted, there was somewhere that could tell you the answer. Two thousand years later, computer scientists borrowed the word. For them, an oracle is a black box: you hand it a problem you cannot solve, and it immediately returns the correct answer. The two oracles share the same fantasy: before moving, verify right and wrong.

This book is about the world after that fantasy breaks.

We almost never verify. We act, and then, sooner or later, we find out; or we never find out at all. We think "everything can be checked" is the normal state because our earliest training comes from an unusually narrow class of tasks: arithmetic, sorting a list, checking a receipt. In those tasks the answer is close at hand, so we mistake them for the shape of the whole world. But once we leave that narrow door, verification immediately becomes a luxury. You can verify seven times eight. You cannot verify, before saying "I do," that the marriage will last; before release, that a codebase has no bugs; before committing yourself, that a theory is true, a company is healthy, or a decision is right. Most consequential actions step onto unverified ground. The oracle does not answer, and you still have to move.

The usual responses to this condition are lament or pretense. Those who lament say that if nothing can be made certain, then every judgment is mere guesswork. Those who pretend build themselves a false oracle: they enthrone a measurable number and act as if it were the unmeasurable truth. This book does neither. It asks a more interesting question: when the oracle is absent, what do capable people actually do: scientists, engineers, mathematicians, and governors?

If you pursue that question across enough fields, you run into a surprising observation. Although the sources of unverifiability differ wildly, capable responses keep converging on the same small set.

That observation gives the book its two-layer structure. Hold it in mind, because everything later hangs from it.

The first layer: the problem is heterogeneous. Beneath the sentence "I cannot check it" lie five structurally different situations. Some have no decision procedure in principle (undecidable). Some have a procedure, but its cost explodes (intractable). Some hide the relevant state from you (partially observable). Some could be verified in principle, but you lack the time, computation, or samples (budget-constrained). Some contain an opposing system actively frustrating your check (adversarial). Treating these five as the same is the most common mistake in this territory. Part I pulls them apart.

The second layer: responses converge. Whatever face the problem wears, capable actors repeatedly reach for the same few things: replace an unmeasurable target with a measurable proxy; prove a bound on a slice you can inspect; spend expensive checks where they carry the most information; introduce an external judge; shrink the blast radius of failure; calibrate residual risk as a probability; move checking from before the fact to after it; and use multiple independent judgments to cancel single-point error. This book calls these the eight moves, and argues that they can be gathered under four more basic levers. Part II enters four concrete sites and lets those moves appear inside their local vocabularies. Part III then lifts each move out, cleans it, names it, and lays out a cross-domain table. That table is the real payload of the book. Part IV asks why these moves, and not others.

One candid reservation has to stand at the front. Is this convergence a law, in the sense that something forces every bounded actor toward these moves? Or is it only a strong empirical pattern: something we keep seeing, but cannot prove must be so? At the moment I do not have evidence that it is a law. What this book delivers is a bounded conjecture, stated plainly, together with a shared vocabulary that can connect many fields. It is not a theorem. Chapter 14 confronts this directly.

That creates an unavoidable and fitting recursion. A book about how to act under unverifiability cannot verify its own central claim. So it can only do what it describes throughout: state a calibrated belief, draw the boundaries of the claim, invite refutation, and proceed anyway. This book will practice the methods it studies. If it is right, that self-demonstration is not a defect. It is the only honest way to write it.

One image to end on; the afterword will return to it. A ship changes course in heavy fog. The captain has charts, a compass, and estimates of the current. She does not have eyes that can see through the fog. She cannot verify, before turning the rudder, whether a reef lies ahead. The fog will not lift. The oracle will not come. But sailing cannot stop for that reason. This book wants to understand not how to wait for the fog to clear, but how a good captain actually steers inside it.

Next chapter: 1. The Luxury of Verification →

References

Waypoints: 1. historical scientific judgment; 2. theoretically studied material; 3. how science progresses; 4. how to live in an unverifiable world. This section was checked source by source.

  1. H. A. Simon (1969). The Sciences of the Artificial. MIT Press. [2][4] Simon distinguishes natural science from "the sciences of the artificial" and argues that design is a discipline for coping with complex environments under bounded rationality. His ideas of near-decomposability, hierarchy, and satisficing provide a background for this book's central stance: actors do not verify everything; under limits of computation and information, they design responses that are good enough.
  2. F. H. Knight (1921). Risk, Uncertainty and Profit. Houghton Mifflin. [2] Knight draws the influential line between risk, where probabilities are known and measurable, and true uncertainty, where even the probability distribution is unavailable. He then attributes entrepreneurial profit to bearing the latter. The distinction is one conceptual source for this book's use of "unverifiability."
  3. N. N. Taleb (2007). The Black Swan: The Impact of the Highly Improbable. Random House. [2][4] Taleb argues that rare, hard-to-foresee, high-impact events dominate history and markets, while conventional bell-curve statistics systematically underestimate them. His diagnosis of prediction's limits matters here: when tail events cannot be verified in advance, changing one's exposure to surprise is often more useful than pursuing precise forecasts.
  4. W. C. Wimsatt (2007). Re-Engineering Philosophy for Limited Beings: Piecewise Approximations to Reality. Harvard University Press. [2][3][4] Wimsatt argues that limited beings cannot possess complete truth. They rely on biased but useful heuristics, robustness analysis, and piecewise approximations to reality. This is a philosophical counterpart to the chapter's main claim and especially to the role of robustness and multiple independent routes of support.
  5. J. M. Keynes (1921). A Treatise on Probability. Macmillan. [2] Keynes understands probability as a logical relation between propositions: the rational degree of belief given evidence. He notes that many probabilities cannot be precisely numbered and may not even be comparable. His notion of the "weight" of evidence reminds us that when evidence is thin, quantified confidence may itself be unwarranted.
  6. L. J. Savage (1954). The Foundations of Statistics. Wiley. [2] Savage gives subjective expected utility a set of axiomatic foundations: if a person's preferences satisfy certain consistency requirements, their choices can be represented as maximizing expected utility under a subjective probability. It is the baseline for later disputes about rational choice under unverifiable outcomes.
  7. D. Ellsberg (1961). "Risk, Ambiguity, and the Savage Axioms." Quarterly Journal of Economics, 75(4), 643-669. [2] Ellsberg's urn experiments show that people systematically prefer known probabilities to unknown ones. This ambiguity aversion violates Savage's axioms and cannot be reconciled by a single subjective probability. It gives experimental force to the Knightian distinction.
  8. H. A. Simon (1955). "A Behavioral Model of Rational Choice." Quarterly Journal of Economics, 69(1), 99-118. [2][4] Simon introduces bounded rationality and satisficing: actors constrained by cognition and information do not enumerate all options and optimize globally. They set an aspiration level and stop when they find an option that meets it. In unverifiable settings, "good enough" is often a rational form, not a failure.
  9. H. A. Simon (1947). Administrative Behavior: A Study of Decision-Making Processes in Administrative Organization. Macmillan. [2][4] Simon treats organizations as decision structures that amplify individual bounded rationality. Organizations set premises, divide responsibilities, and build routines so members can act under incomplete information. This book extends bounded rationality from individuals to institutions.
  10. A. Tversky and D. Kahneman (1974). "Judgment under Uncertainty: Heuristics and Biases." Science, 185(4157), 1124-1131. [2] Tversky and Kahneman show that people estimate probabilities through a small set of heuristics, such as representativeness, availability, and anchoring. These shortcuts often work but also produce predictable biases. The paper is a starting point for understanding where human judgment is reliable and where it fails.
  11. D. Kahneman and A. Tversky (1979). "Prospect Theory: An Analysis of Decision under Risk." Econometrica, 47(2), 263-291. [2][4] Prospect theory models real choice through a value function relative to a reference point and nonlinear weighting of probabilities. People weigh losses more heavily than equal gains and distort small and large probabilities. It is a descriptive correction to classical expected utility.
  12. D. Kahneman (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux. [2][4] Kahneman synthesizes decades of research on judgment and decision-making through the dual-process frame of System 1 and System 2. The book helps build a general picture of cognitive limits and why even experts need external correction mechanisms.
  13. G. Gigerenzer and D. G. Goldstein (1996). "Reasoning the Fast and Frugal Way: Models of Bounded Rationality." Psychological Review, 103(4), 650-669. [2][4] Gigerenzer and Goldstein argue that simple rules using few cues and stopping early can, in real environments, match or outperform more complex statistical models. Their work counters the idea that heuristics are merely biases and asks why simplicity can be effective.
  14. G. Gigerenzer, P. M. Todd, and the ABC Research Group (1999). Simple Heuristics That Make Us Smart. Oxford University Press. [2][4] This collection develops the adaptive toolbox program: minds carry a set of simple heuristics tuned to particular environments, and their success depends on ecological rationality. The cases show how simple rules can make robust decisions under limited information.
  15. F. A. Hayek (1945). "The Use of Knowledge in Society." American Economic Review, 35(4), 519-530. [2][4] Hayek argues that economically relevant knowledge is dispersed, local, and tied to particular circumstances. No central planner can gather it all; prices coordinate it in decentralized form. This reveals one source of unverifiability: the relevant information is never fully held by a single actor.
  16. M. Polanyi (1958). Personal Knowledge: Towards a Post-Critical Philosophy. Routledge and Kegan Paul. [1][3][4] Polanyi argues that all knowing contains tacit knowledge and personal commitment. Fully objective, fully formalized knowledge is an illusion. His account explains why scientific judgment cannot be replaced entirely by rules.
  17. P. E. Meehl (1954). Clinical versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. University of Minnesota Press. [1][4] Meehl reviews evidence showing that simple statistical or actuarial predictions often match or exceed clinical expert judgment. It is classic evidence for outsourcing judgment to checkable rules, while warning that expert confidence and expert accuracy may diverge.
  18. D. A. Schon (1983). The Reflective Practitioner: How Professionals Think in Action. Basic Books. [1][4] Schon describes reflection-in-action: professionals in ambiguous and unique situations do not simply apply theory. They converse with the situation, act, and reframe the problem. This captures a professional ability that cannot be verified in advance.
  19. G. A. Klein (1998). Sources of Power: How People Make Decisions. MIT Press. [1][4] Klein's field studies of firefighters, nurses, and other experts lead to the recognition-primed decision model. Experienced people under time pressure often generate a workable action by recognizing a pattern, then mentally simulating it. This helps explain when expert intuition can be reliable.
  20. P. E. Tetlock (2005). Expert Political Judgment: How Good Is It? How Can We Know? Princeton University Press. [1][4] Tetlock tracks political and economic experts over many years and finds that their average forecasts often fall short of simple extrapolation. Cognitive style explains more than credentials: pluralistic, self-questioning "foxes" outperform single-theory "hedgehogs." The book puts expert judgment under scoreable test.
  21. P. E. Tetlock and D. Gardner (2015). Superforecasting: The Art and Science of Prediction. Crown. [1][4] This book extends Tetlock's forecasting tournament work and describes "superforecasters": people who decompose problems, assign scoreable probabilities, update frequently, and use team correction. Forecasting appears not as a gift but as a learnable practice.
  22. N. N. Taleb (2001). Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets. Texere. [2][4] Taleb argues that people regularly mistake random outcomes for skill or necessity, especially in markets, where survivors are treated as masters. The book warns that success itself does not verify a judgment when causal structure is uncertain.
  23. N. N. Taleb (2012). Antifragile: Things That Gain from Disorder. Random House. [4] Taleb introduces antifragility: some systems do not merely survive volatility, but benefit from it. In an unpredictable world, he argues, one should preserve optionality and cap downside risk. This directly connects to shrinking the blast radius of failure.
  24. C. E. Lindblom (1959). "The Science of 'Muddling Through'." Public Administration Review, 19(2), 79-88. [4] Lindblom argues that real public policy is often incremental rather than globally rational: actors make limited changes near the status quo, compare as they go, and remain tied to existing means. This legitimizes cautious correction as a reasonable response to complexity.
  25. K. R. Popper (1959). The Logic of Scientific Discovery. Hutchinson. [3] Popper develops falsificationism: scientific theories cannot be verified as true, only exposed to possible refutation. Falsifiability, not verification, marks the boundary between science and non-science. The book is one philosophical source for the present argument.
  26. T. S. Kuhn (1962). The Structure of Scientific Revolutions. University of Chicago Press. [1][3] Kuhn argues that science alternates between normal science under a paradigm and revolutionary shifts after accumulated anomalies produce crisis. Judgment and community matter; progress is not a simple linear accumulation of verified truths.
  27. W. V. Quine (1951). "Two Dogmas of Empiricism." The Philosophical Review, 60(1), 20-43. [3] Quine attacks the analytic-synthetic divide and the idea that each statement faces experience alone. Beliefs meet evidence as a web; any statement can be held if changes are made elsewhere. This is a classic argument for the underdetermination of theory by evidence.
  28. P. Duhem (1954). The Aim and Structure of Physical Theory. Princeton University Press. [3] Duhem argues that physical experiments never test a single hypothesis in isolation. They test a whole bundle of theory and auxiliary assumptions, so a failed prediction does not uniquely identify what is wrong. This is central to understanding why science does not rest on decisive single verifications.
  29. I. Hacking (1983). Representing and Intervening: Introductory Topics in the Philosophy of Natural Science. Cambridge University Press. [3] Hacking shifts attention from representation to intervention: when we can reliably manipulate an entity to intervene in the world, we gain reason to believe in it. His experimental realism reminds us that verification is not only passive observation; it is also action.
← Cover1. The Luxury of Verification →

Open with WeChat

Scan this page in WeChat, then use WeChat's share menu.