Bayesian Optimization
Part III: Bayesian Optimization
中文

Bayesian Optimization

With a surrogate that knows what it does not know, optimization becomes a sequence of decisions: given everything observed so far, where should the next, expensive evaluation go? This part builds the loop that makes those decisions, derives the classic answers to the question, and asks how to tell whether an optimizer is any good.

Acquisition functions are the heart of the part: probability of improvement, expected improvement, upper confidence bounds, Thompson sampling, knowledge gradient, and entropy search, each a different answer to "what is the next observation worth?". Regret and bandits give the theory a vocabulary. The last two chapters leave the textbook for practice: noise, batches, constraints, many dimensions, software, and the fields where Bayesian optimization has paid off.

The part assumes Gaussian process regression (Chapter 8).

Chapters in this part

  1. 11 The Bayesian Optimization Loop

    The problem Bayesian optimization solves, the loop that solves it (fit a surrogate, maximize an acquisition function, evaluate, update), the trade-off between exploring and exploiting that every acquisition function must strike, how to choose the first points, and where the method came from.

  2. 12 Acquisition Functions

    Probability of improvement, expected improvement, upper confidence bounds, Thompson sampling, the knowledge gradient, and entropy search: each a different answer to 'what is the next evaluation worth?', derived and compared on the same posterior in one and two dimensions, followed by how the acquisition function itself is maximized when the domain has many dimensions.

  3. 13 Regret, Bandits, and Guarantees

    How optimizers are scored: simple and cumulative regret, the multi-armed bandit and its classic algorithms, the Lai-Robbins lower bound, GP-UCB's regret bound through the maximum information gain, and what such guarantees do and do not promise in practice.

  4. 14 Bayesian Optimization in Practice

    The decisions a working Bayesian optimization system makes beyond the textbook loop: how to set up the surrogate, what to report under noise, how to choose batches, constraints and safety, several objectives, many dimensions, costs and stopping, and which software implements what.

  5. 15 Where Bayesian Optimization Works

    Hyperparameters, laboratories, robots, engineering designs, online experiments, and people: where Bayesian optimization has paid off and why, and how it relates to its neighbors in active learning, experimental design, bandits, reinforcement learning, and evolution strategies.

References for Part III

129 works cited across this part's chapters.

  1. Abbasi-Yadkori, Y., Pál, D., and Szepesvári, C. (2011). Improved Algorithms for Linear Stochastic Bandits. Advances in Neural Information Processing Systems. Ch. 13
  2. Adesiji, A. D., Wang, J., Kuo, C.-S., and Brown, K. A. (2026). Benchmarking self-driving labs. Digital Discovery. Ch. 15
  3. Agrawal, S., and Goyal, N. (2012). Analysis of Thompson Sampling for the Multi-armed Bandit Problem. Conference on Learning Theory. Ch. 13
  4. Agrawal, S., and Goyal, N. (2013). Further Optimal Regret Bounds for Thompson Sampling. International Conference on Artificial Intelligence and Statistics. Ch. 13
  5. Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019). Optuna: A Next-generation Hyperparameter Optimization Framework. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2019). Ch. 14
  6. Ament, S., Daulton, S., Eriksson, D., Balandat, M., and Bakshy, E. (2023). Unexpected Improvements to Expected Improvement for Bayesian Optimization. Advances in Neural Information Processing Systems 36 (NeurIPS 2023). Ch. 12 Ch. 14
  7. Attia, P. M., Grover, A., Jin, N., Severson, K. A., Markov, T. M., Liao, Y.-H., … Chueh, W. C. (2020). Closed-Loop Optimization of Fast-Charging Protocols for Batteries with Machine Learning. Nature. Ch. 15
  8. Auer, P., Cesa-Bianchi, N., and Fischer, P. (2002). Finite-time Analysis of the Multiarmed Bandit Problem. Machine Learning. Ch. 13
  9. AutoML.org (2026). smac 2.4.1. PyPI. software Ch. 14
  10. Balandat, M., Karrer, B., Jiang, D. R., Daulton, S., Letham, B., Wilson, A. G., and Bakshy, E. (2020). BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Ch. 11 Ch. 12 Ch. 14
  11. Bergstra, J., and Bengio, Y. (2012). Random Search for Hyper-Parameter Optimization. Journal of Machine Learning Research. Ch. 11 Ch. 15
  12. Bergstra, J., Bardenet, R., Bengio, Y., and Kégl, B. (2011). Algorithms for Hyper-Parameter Optimization. Advances in Neural Information Processing Systems 24 (NeurIPS 2011). Ch. 14 Ch. 15
  13. Berkenkamp, F., Schoellig, A. P., and Krause, A. (2016). Safe Controller Optimization for Quadrotors with Gaussian Processes. IEEE International Conference on Robotics and Automation (ICRA 2016). Ch. 15
  14. Berkenkamp, F., Schoellig, A. P., and Krause, A. (2019). No-Regret Bayesian Optimization with Unknown Hyperparameters. Journal of Machine Learning Research. Ch. 13
  15. Brochu, E., de Freitas, N., and Ghosh, A. (2007). Active Preference Learning with Discrete Choice Data. Advances in Neural Information Processing Systems. Ch. 11
  16. Brochu, E., Cora, V. M., and de Freitas, N. (2010). A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning. arXiv preprint. preprint Ch. 11 Ch. 12
  17. Bubeck, S., Munos, R., and Stoltz, G. (2009). Pure Exploration in Multi-armed Bandits Problems. Algorithmic Learning Theory (ALT 2009). Ch. 13
  18. Bull, A. D. (2011). Convergence Rates of Efficient Global Optimization Algorithms. Journal of Machine Learning Research. Ch. 13
  19. Burger, B., Maffettone, P. M., Gusev, V. V., Aitchison, C. M., Bai, Y., Wang, X., … Cooper, A. I. (2020). A Mobile Robotic Chemist. Nature. Ch. 15
  20. Calandra, R., Seyfarth, A., Peters, J., and Deisenroth, M. P. (2016). Bayesian Optimization for Learning Gaits under Uncertainty. Annals of Mathematics and Artificial Intelligence. Ch. 15
  21. Chaloner, K., and Verdinelli, I. (1995). Bayesian Experimental Design: A Review. Statistical Science. Ch. 15
  22. Chen, Y., Huang, A., Wang, Z., Antonoglou, I., Schrittwieser, J., Silver, D., and de Freitas, N. (2018). Bayesian Optimization in AlphaGo. preprint Ch. 15
  23. Chowdhury, S. R., and Gopalan, A. (2017). On Kernelized Multi-armed Bandits. International Conference on Machine Learning. Ch. 12 Ch. 13
  24. Clark, C. E. (1961). The Greatest of a Finite Set of Random Variables. Operations Research. Ch. 12
  25. Cowen-Rivers, A. I., Lyu, W., Tutunov, R., Wang, Z., Grosnit, A., Griffiths, R. R., … Bou-Ammar, H. (2022). HEBO: An Empirical Study of Assumptions in Bayesian Optimisation. Journal of Artificial Intelligence Research. Ch. 14
  26. Cully, A., Clune, J., Tarapore, D., and Mouret, J.-B. (2015). Robots That Can Adapt like Animals. Nature. Ch. 15
  27. Daulton, S., Balandat, M., and Bakshy, E. (2020). Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Ch. 14
  28. Daulton, S., Balandat, M., and Bakshy, E. (2021). Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement. Advances in Neural Information Processing Systems 34 (NeurIPS 2021). Ch. 14
  29. Desautels, T., Krause, A., and Burdick, J. W. (2014). Parallelizing Exploration-Exploitation Tradeoffs in Gaussian Process Bandit Optimization. Journal of Machine Learning Research. Ch. 14
  30. Ding, Y., Kim, M., Kuindersma, S., and Walsh, C. J. (2018). Human-in-the-Loop Optimization of Hip Assistance with a Soft Exosuit during Walking. Science Robotics. Ch. 15
  31. Doumont, C., Fan, D., Maus, N., Gardner, J. R., Moss, H., and Pleiss, G. (2026). We Still Don't Understand High-Dimensional Bayesian Optimization. AISTATS 2026 (best student paper). Ch. 14
  32. Dragonfly developers (2022). dragonfly-opt 0.1.7. PyPI. software Ch. 14
  33. Duris, J., Kennedy, D., Hanuka, A., Shtalenkova, J., Edelen, A., Baxevanis, P., … Ratner, D. (2020). Bayesian Optimization of a Free-Electron Laser. Physical Review Letters. Ch. 15
  34. Emmerich, M. T. M., Giannakoglou, K. C., and Naujoks, B. (2006). Single- and Multiobjective Evolutionary Optimization Assisted by Gaussian Random Field Metamodels. IEEE Transactions on Evolutionary Computation. Ch. 14
  35. Eriksson, D., and Jankowiak, M. (2021). High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces. Uncertainty in Artificial Intelligence. Ch. 14
  36. Eriksson, D., Pearce, M., Gardner, J., Turner, R. D., and Poloczek, M. (2019). Scalable Global Optimization via Local Bayesian Optimization. Advances in Neural Information Processing Systems 32 (NeurIPS 2019). Ch. 12 Ch. 14 Ch. 15
  37. Falkner, S., Klein, A., and Hutter, F. (2018). BOHB: Robust and Efficient Hyperparameter Optimization at Scale. International Conference on Machine Learning. Ch. 14
  38. Forrester, A. I. J., Sóbester, A., and Keane, A. J. (2008). Engineering Design via Surrogate Modelling: A Practical Guide. Wiley. Ch. 15
  39. Frazier, P. I. (2018). A Tutorial on Bayesian Optimization. arXiv. preprint Ch. 11 Ch. 12 Ch. 14
  40. Frazier, P. I., Powell, W. B., and Dayanik, S. (2008). A Knowledge-Gradient Policy for Sequential Information Collection. SIAM Journal on Control and Optimization. Ch. 12
  41. Frazier, P., Powell, W., and Dayanik, S. (2009). The Knowledge-Gradient Policy for Correlated Normal Beliefs. INFORMS Journal on Computing. Ch. 11 Ch. 12
  42. Gardner, J. R., Kusner, M. J., Xu, Z., Weinberger, K. Q., and Cunningham, J. P. (2014). Bayesian Optimization with Inequality Constraints. Proceedings of the 31st International Conference on Machine Learning (ICML 2014). Ch. 14
  43. Garnett, R. (2023). Bayesian Optimization. Cambridge University Press. Ch. 11 Ch. 12 Ch. 13 Ch. 14
  44. Gelbart, M. A., Snoek, J., and Adams, R. P. (2014). Bayesian Optimization with Unknown Constraints. Conference on Uncertainty in Artificial Intelligence (UAI 2014). Ch. 14
  45. Ginsbourger, D., Le Riche, R., and Carraro, L. (2010). Kriging Is Well-Suited to Parallelize Optimization. Computational Intelligence in Expensive Optimization Problems. Ch. 14
  46. Golovin, D., Solnik, B., Moitra, S., Kochanski, G., Karro, J., and Sculley, D. (2017). Google Vizier: A Service for Black-Box Optimization. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2017). Ch. 15
  47. González, J., Dai, Z., Hennig, P., and Lawrence, N. (2016). Batch Bayesian Optimization via Local Penalization. International Conference on Artificial Intelligence and Statistics. Ch. 14
  48. Hansen, N., and Ostermeier, A. (2001). Completely Derandomized Self-Adaptation in Evolution Strategies. Evolutionary Computation. Ch. 15
  49. Hennig, P., and Schuler, C. J. (2012). Entropy Search for Information-Efficient Global Optimization. Journal of Machine Learning Research. Ch. 11 Ch. 12
  50. Hernández-Lobato, J. M., Hoffman, M. W., and Ghahramani, Z. (2014). Predictive Entropy Search for Efficient Global Optimization of Black-box Functions. Advances in Neural Information Processing Systems 27 (NeurIPS 2014). Ch. 11 Ch. 12
  51. Hoeffding, W. (1963). Probability Inequalities for Sums of Bounded Random Variables. Journal of the American Statistical Association. Ch. 13
  52. Huawei Noah's Ark Lab (2024). HEBO 0.3.6. PyPI. software Ch. 14
  53. Hutter, F., Hoos, H. H., and Leyton-Brown, K. (2011). Sequential Model-Based Optimization for General Algorithm Configuration. Learning and Intelligent Optimization (LION 5). Ch. 14
  54. Hvarfner, C., Hellsten, E. O., and Nardi, L. (2024). Vanilla Bayesian Optimization Performs Great in High Dimensions. International Conference on Machine Learning. Ch. 12 Ch. 14
  55. Ishibashi, H., Karasuyama, M., Takeuchi, I., and Hino, H. (2023). A stopping criterion for Bayesian optimization by the gap of expected minimum simple regrets. International Conference on Artificial Intelligence and Statistics. Ch. 14
  56. Jones, D. R. (2001). A Taxonomy of Global Optimization Methods Based on Response Surfaces. Journal of Global Optimization. Ch. 12
  57. Jones, D. R., Schonlau, M., and Welch, W. J. (1998). Efficient Global Optimization of Expensive Black-Box Functions. Journal of Global Optimization. Ch. 11 Ch. 12 Ch. 15
  58. Kalinin, S. V., Liu, Y., Biswas, A., Duscher, G., Pratiush, U., Roccapriore, K., Ziatdinov, M., and Vasudevan, R. (2024). Human-in-the-loop: The future of Machine Learning in Automated Electron Microscopy. Microscopy Today. doi:10.1093/mictod/qaad096. Ch. 15
  59. Kanarik, K. J., Osowiecki, W. T., Lu, Y., Talukder, D., Roschewsky, N., Park, S. N., … Gottscho, R. A. (2023). Human–machine collaboration for improving semiconductor process development. Nature. Ch. 15
  60. Kandasamy, K., Schneider, J., and Póczos, B. (2015). High Dimensional Bayesian Optimisation and Bandits via Additive Models. International Conference on Machine Learning. Ch. 14
  61. Kandasamy, K., Dasarathy, G., Schneider, J., and Póczos, B. (2017). Multi-fidelity Bayesian Optimisation with Continuous Approximations. International Conference on Machine Learning. Ch. 14
  62. Kandasamy, K., Krishnamurthy, A., Schneider, J., and Póczos, B. (2018). Parallelised Bayesian Optimisation via Thompson Sampling. International Conference on Artificial Intelligence and Statistics. Ch. 14
  63. Kaufmann, E., Korda, N., and Munos, R. (2012). Thompson Sampling: An Asymptotically Optimal Finite-Time Analysis. Algorithmic Learning Theory (ALT 2012). Ch. 13
  64. Kim, M., Ding, Y., Malcolm, P., Speeckaert, J., Siviy, C. J., Walsh, C. J., and Kuindersma, S. (2017). Human-in-the-Loop Bayesian Optimization of Wearable Device Parameters. PLOS ONE. Ch. 15
  65. Klein, A., Falkner, S., Bartels, S., Hennig, P., and Hutter, F. (2017). Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets. Artificial Intelligence and Statistics. Ch. 14
  66. Knowles, J. (2006). ParEGO: A Hybrid Algorithm with On-line Landscape Approximation for Expensive Multiobjective Optimization Problems. IEEE Transactions on Evolutionary Computation. Ch. 14
  67. Krige, D. G. (1951). A Statistical Approach to Some Basic Mine Valuation Problems on the Witwatersrand. Journal of the Southern African Institute of Mining and Metallurgy. Ch. 11
  68. Kushner, H. J. (1964). A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise. Journal of Basic Engineering. Ch. 11 Ch. 12
  69. Lai, T. L., and Robbins, H. (1985). Asymptotically Efficient Adaptive Allocation Rules. Advances in Applied Mathematics. Ch. 13
  70. Lattimore, T., and Szepesvári, C. (2020). Bandit Algorithms. Cambridge University Press. doi:10.1017/9781108571401. Ch. 13 Ch. 15
  71. Letham, B., Karrer, B., Ottoni, G., and Bakshy, E. (2019). Constrained Bayesian Optimization with Noisy Experiments. Bayesian Analysis. Ch. 14 Ch. 15
  72. Letham, B., Calandra, R., Rai, A., and Bakshy, E. (2020). Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization. Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Ch. 14
  73. Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., and Talwalkar, A. (2018). Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization. Journal of Machine Learning Research. Ch. 14
  74. Lin, Z. J., Astudillo, R., Frazier, P., and Bakshy, E. (2022). Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes. International Conference on Artificial Intelligence and Statistics. Ch. 14 Ch. 15
  75. Lindauer, M., Eggensperger, K., Feurer, M., Biedenkapp, A., Deng, D., Benjamins, C., … Hutter, F. (2022). SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization. Journal of Machine Learning Research. Ch. 14
  76. Lindley, D. V. (1956). On a Measure of the Information Provided by an Experiment. The Annals of Mathematical Statistics. Ch. 15
  77. Lizotte, D., Wang, T., Bowling, M., and Schuurmans, D. (2007). Automatic Gait Optimization with Gaussian Process Regression. Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007). Ch. 15
  78. Loeppky, J. L., Sacks, J., and Welch, W. J. (2009). Choosing the Sample Size of a Computer Experiment: A Practical Guide. Technometrics. Ch. 11
  79. MacKay, D. J. C. (1992). Information-Based Objective Functions for Active Data Selection. Neural Computation. Ch. 15
  80. Matheron, G. (1963). Principles of Geostatistics. Economic Geology. Ch. 11
  81. McKay, M. D., Beckman, R. J., and Conover, W. J. (1979). A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code. Technometrics. Ch. 11
  82. Meta Platforms, Inc. (2026a). ax-platform release history. PyPI. software Ch. 14
  83. Meta Platforms, Inc. (2026e). BoTorch CHANGELOG. GitHub. software Ch. 14
  84. Meta Platforms, Inc. (2026i). botorch release history. PyPI. software Ch. 14
  85. Meta Platforms, Inc. (2026k). botorch/models/utils/gpytorch_modules.py. GitHub. software Ch. 14
  86. Močkus, J. (1975). On Bayesian Methods for Seeking the Extremum. Optimization Techniques IFIP Technical Conference. Ch. 11 Ch. 12
  87. Olson, M., Santorella, E., Tiao, L. C., Cakmak, S., Garrard, M., Daulton, S., … Bakshy, E. (2025). Ax: A Platform for Adaptive Experimentation. International Conference on Automated Machine Learning. Ch. 14 Ch. 15
  88. Optuna contributors (2026). optuna.samplers.GPSampler, Optuna 5.0.0 documentation. Read the Docs. software Ch. 14
  89. Optuna developers (2026a). optuna 5.0.0. PyPI. software Ch. 14
  90. Papenmeier, L., Poloczek, M., and Nardi, L. (2025b). Understanding High-Dimensional Bayesian Optimization. ICML 2025, PMLR 267:47902-47923. Ch. 14
  91. Picheny, V., Wagner, T., and Ginsbourger, D. (2013). A Benchmark of Kriging-Based Infill Criteria for Noisy Optimization. Structural and Multidisciplinary Optimization. Ch. 14
  92. Rahimi, A., and Recht, B. (2007). Random Features for Large-Scale Kernel Machines. Advances in Neural Information Processing Systems 20 (NeurIPS 2007). Ch. 12
  93. Robbins, H. (1952). Some Aspects of the Sequential Design of Experiments. Bulletin of the American Mathematical Society. Ch. 13
  94. Russo, D., and Van Roy, B. (2014). Learning to Optimize via Posterior Sampling. Mathematics of Operations Research. Ch. 12
  95. Russo, D. J., Van Roy, B., Kazerouni, A., Osband, I., and Wen, Z. (2018). A Tutorial on Thompson Sampling. Foundations and Trends in Machine Learning. Ch. 13
  96. Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P. (1989). Design and Analysis of Computer Experiments. Statistical Science. Ch. 11
  97. Scarlett, J., Bogunovic, I., and Cevher, V. (2017). Lower Bounds on Regret for Noisy Gaussian Process Bandit Optimization. Conference on Learning Theory. Ch. 13
  98. scikit-optimize contributors (2024). scikit-optimize 0.10.2. PyPI; the GitHub repository is archived. software Ch. 14
  99. Secondmind Labs (2026). trieste 4.6.0. PyPI. software Ch. 14
  100. Settles, B. (2009). Active Learning Literature Survey. University of Wisconsin–Madison. non-peer-reviewed Ch. 15
  101. Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., and de Freitas, N. (2016). Taking the Human Out of the Loop: A Review of Bayesian Optimization. Proceedings of the IEEE. Ch. 11 Ch. 15
  102. SheffieldML (2023). GPyOpt (archived). GitHub. software Ch. 14
  103. Shields, B. J., Stevens, J., Li, J., Parasram, M., Damani, F., Alvarado, J. I. M., … Doyle, A. G. (2021). Bayesian reaction optimization as a tool for chemical synthesis. Nature. Ch. 11 Ch. 15
  104. Snoek, J., Larochelle, H., and Adams, R. P. (2012). Practical Bayesian Optimization of Machine Learning Algorithms. Advances in Neural Information Processing Systems 25 (NeurIPS 2012). Ch. 11 Ch. 12 Ch. 14 Ch. 15
  105. Snoek, J., Swersky, K., Zemel, R., and Adams, R. (2014). Input Warping for Bayesian Optimization of Non-Stationary Functions. International Conference on Machine Learning. Ch. 14
  106. Sobol', I. M. (1967). On the Distribution of Points in a Cube and the Approximate Evaluation of Integrals. USSR Computational Mathematics and Mathematical Physics. Ch. 11
  107. Srinivas, N., Krause, A., Kakade, S. M., and Seeger, M. (2010). Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design. ICML 2010. Ch. 11 Ch. 12 Ch. 13
  108. Sui, Y., Gotovos, A., Burdick, J., and Krause, A. (2015). Safe Exploration for Optimization with Gaussian Processes. Proceedings of the 32nd International Conference on Machine Learning (ICML 2015). Ch. 14
  109. Sui, Y., Zhuang, V., Burdick, J., and Yue, Y. (2018b). Stagewise Safe Bayesian Optimization with Gaussian Processes. International Conference on Machine Learning. Ch. 14
  110. Sutton, R. S., and Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press. Ch. 15
  111. Swersky, K., Snoek, J., and Adams, R. P. (2013). Multi-Task Bayesian Optimization. Advances in Neural Information Processing Systems 26 (NeurIPS 2013). Ch. 14
  112. Takagi, H. (2001). Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation. Proceedings of the IEEE. Ch. 15
  113. Thompson, W. R. (1933). On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two Samples. Biometrika. Ch. 11 Ch. 12 Ch. 13
  114. Thornton, C., Hutter, F., Hoos, H. H., and Leyton-Brown, K. (2013). Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms. Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2013). Ch. 15
  115. Turner, R., Eriksson, D., McCourt, M., Kiili, J., Laaksonen, E., Xu, Z., and Guyon, I. (2021). Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning: Analysis of the Black-Box Optimization Challenge 2020. NeurIPS 2020 Competition and Demonstration Track. Ch. 15
  116. Vakili, S., Khezeli, K., and Picheny, V. (2021a). On Information Gain and Regret Bounds in Gaussian Process Bandits. International Conference on Artificial Intelligence and Statistics. Ch. 13
  117. Villemonteix, J., Vazquez, E., and Walter, E. (2009). An Informational Approach to the Global Optimization of Expensive-to-Evaluate Functions. Journal of Global Optimization. Ch. 12
  118. Wang, Z., and Jegelka, S. (2017). Max-value Entropy Search for Efficient Bayesian Optimization. Proceedings of the 34th International Conference on Machine Learning (ICML 2017). Ch. 11 Ch. 12
  119. Wang, Z., Hutter, F., Zoghi, M., Matheson, D., and de Freitas, N. (2016). Bayesian Optimization in a Billion Dimensions via Random Embeddings. Journal of Artificial Intelligence Research. Ch. 14
  120. Weichert, D., Ernis, G., Worthmann, M., Ryzko, P., and Seifert, L. (2025). When Less is More: A Story of Failing Bayesian Optimization Due to Additional Expert Knowledge. arXiv. preprint Ch. 15
  121. Wilson, J. T. (2024). Stopping Bayesian Optimization with Probabilistic Regret Bounds. NeurIPS 2024. Ch. 14
  122. Wilson, J. T., Hutter, F., and Deisenroth, M. P. (2018). Maximizing Acquisition Functions for Bayesian Optimization. Advances in Neural Information Processing Systems 31 (NeurIPS 2018). Ch. 12 Ch. 14
  123. Wilson, J. T., Borovitskiy, V., Terenin, A., Mostowski, P., and Deisenroth, M. P. (2020). Efficiently Sampling Functions from Gaussian Process Posteriors. Proceedings of the 37th International Conference on Machine Learning (ICML 2020). Ch. 12
  124. Wu, J., Toscano-Palmerin, S., Frazier, P. I., and Wilson, A. G. (2019). Practical Multi-fidelity Bayesian Optimization for Hyperparameter Tuning. Uncertainty in Artificial Intelligence (UAI 2019). Ch. 14
  125. Xie, Q., Astudillo, R., Frazier, P. I., Scully, Z., and Terenin, A. (2024). Cost-aware Bayesian Optimization via the Pandora's Box Gittins Index. NeurIPS 2024. Ch. 14
  126. Xie, Q., Cai, L., Terenin, A., Frazier, P. I., and Scully, Z. (2026). Cost-aware Stopping for Bayesian Optimization. International Conference on Machine Learning. Ch. 14
  127. Xu, W., Wang, W., Jiang, Y., Svetozarevic, B., and Jones, C. (2024b). Principled Preferential Bayesian Optimization. International Conference on Machine Learning. Ch. 13
  128. Xu, Z., Wang, H., Phillips, J. M., and Zhe, S. (2025b). Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization. ICLR 2025 (oral). Ch. 14
  129. Zhang, J., Fiers, P., Witte, K. A., Jackson, R. W., Poggensee, K. L., Atkeson, C. G., and Collins, S. H. (2017). Human-in-the-loop optimization of exoskeleton assistance during walking. Science. Ch. 15