Changkun OuBayesian OptimizationFrom First Principles to Human Preferences
Bayesian Optimization
From First Principles to Human Preferences
Some functions are expensive to evaluate and impossible to write down: the
accuracy of a model after a day of training, the comfort of an exoskeleton
after a minute of walking, the look of a design that only a person can judge.
Bayesian optimization finds good inputs for such functions with few
evaluations. It keeps a probabilistic model of what every untried input might
return, and spends each evaluation where the model expects to learn the most.
When the only measurement is a person saying "this one, not that one", the same
idea becomes preferential Bayesian optimization, and the person becomes part of
the system being studied.
This book teaches both from the beginning: the probability and linear algebra
a software engineer may not have used since school, Gaussian processes and the
analysis behind their kernels, the optimization loop, and learning from
comparisons. It works four real problems
end to end, then follows the research through 2026: what has been proved, what
happened with real people, and what psychology, economics, neuroscience, and
philosophy say about whether a preference is there to be found. Its argument
is that the algorithms are now mature and the hard part has moved to
measurement: what a single comparison measures, and what asking does to the
person who answers.
An optimizer at work on a function it cannot see (dashed). Blue is what it believes, the band is how unsure it is, and orange is where it looks next.
Every chapter has figures you can change, derivations you can step through,
and references attached to the section that uses them. In some figures you are
the person being optimized.
The book is open source on
GitHub. If you find an error, a number
that does not match its source, or a passage that is hard to follow, please
open an issue; every page also
has a link at the bottom that does this for that page.
