Bayesian Optimization
Part VII: People in the Loop
中文

People in the Loop

Preferential optimization is done with people, and to them. This part collects what happened when the methods met real users: in interactive design tools, where studies measured not only whether people reached good designs but whether they felt the designs were theirs; in exoskeletons, prostheses, hearing aids, and neural implants, where the stakes are physical; and in buildings, vehicles, laboratories, and industry, where the tools are ready but few real people have used them.

These studies are where the book's central claim gets its evidence. The algorithms work well enough; what limits them is measurement (Section 45.1). Feedback in real loops is unstable, people answer differently from the simulated users that methods are tuned on, and the way the optimizer asks changes the person's experience of the task, so a session is an intervention as well as an estimate. Well-controlled studies more often find "the same result with fewer iterations or less cost" than "a better result", and simple alternatives, including letting people tune devices themselves, sometimes do as well. The chapters report each study with its design and numbers, so that you can judge how far each finding carries.

The part assumes Chapter 19; it can be read before the frontier chapters.

Chapters in this part

  1. 32 Interactive Design and Human-Computer Interaction

    What interactive design studies from 2017 to 2026 found when the person in the loop was real: six research lines, a consistent trade-off between agency and performance, unstable feedback in real sessions, evidence on feedback forms, population priors, expertise, explanations, and what happens against strong baselines.

  2. 33 Wearable Robots, Health, and Assistive Technology

    Exoskeletons, prostheses, hearing aids, neural implants, and robot controllers tuned from a person's preferences: who took part, what was measured, how the results were validated, and how algorithm tuning compares with letting people tune devices themselves.

  3. 34 Built Environments, Science, and Industry

    Buildings, vehicles, expert-guided science, and industry: where preferential optimization has tools and methods but few real people, how the evidence compares across every application domain, and which common claims about it hold up.

References for Part VII

176 works cited across this part's chapters.

  1. Abdelrahman, M., and Miller, C. (2022). Targeting occupant feedback using digital twins: Adaptive spatial-temporal thermal preference sampling to optimize personal comfort models. Building and Environment 218. Ch. 34
  2. Adachi, M., Planden, B., Howey, D. A., Osborne, M. A., Orbell, S., Ares, N., Muandet, K., and Chau, S. L. (2024). Looping in the Human Collaborative and Explainable Bayesian Optimization. AISTATS 2024. Ch. 32 Ch. 34
  3. Adachi, M., Chau, S. L., Xu, W., Singh, A., Osborne, M. A., and Muandet, K. (2025). Bayesian Optimization for Building Social-Influence-Free Consensus. arXiv. preprint Ch. 34
  4. Alili, A., Nalam, V., Li, M., Liu, M., Feng, J., Si, J., and Huang, H. (2023). A Novel Framework to Facilitate User Preferred Tuning for a Robotic Knee Prosthesis. IEEE Transactions on Neural Systems and Rehabilitation Engineering. Ch. 33
  5. Amsterdam UMC (2008). Personalization of Hearing Aids through Bayesian Preference Elicitation. Trial registry, onderzoekmetmensen.nl. non-peer-reviewed Ch. 33
  6. Arens, P., Quirk, D. A., Pan, W., Yacoby, Y., Doshi-Velez, F., and Walsh, C. J. (2025). Preference-based assistance optimization for lifting and lowering with a soft back exosuit. Science Advances. doi:10.1126/sciadv.adu2099. Ch. 33 Ch. 34
  7. Arun Kumar A V, Shilton, A., Gupta, S., Rana, S., Greenhill, S., and Venkatesh, S. (2024). Enhanced Bayesian Optimization via Preferential Modeling of Abstract Properties. ECML PKDD 2024. Ch. 34
  8. Astudillo, R., Lin, Z. J., Bakshy, E., and Frazier, P. (2023). qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimization. International Conference on Artificial Intelligence and Statistics. Ch. 34
  9. Astudillo, R., Li, K., Tucker, M., Cheng, C. X., Ames, A. D., and Yue, Y. (2025). Preferential Multi-Objective Bayesian Optimization. Transactions on Machine Learning Research. Ch. 33 Ch. 34
  10. Avelino, R. M., Sevastjanova, R., Van Mele, T., Block, P., and El-Assady, M. (2026). Creativity from Friction: Human-AI Interaction for Exploratory Structural Design. ICML 2026 Workshop on Human-AI Co-Creativity. workshop paper Ch. 32
  11. Awalgaonkar, N., Bilionis, I., Liu, X., Karava, P., and Tzempelikos, A. (2019). Learning Personalized Thermal Preferences via Bayesian Active Learning with Unimodality Constraints. arXiv. preprint Ch. 34
  12. Baek, S., Park, S., and Kim, D. (2026). Context-Continuous Preference Learning for Exoskeleton Personalization. arXiv. preprint Ch. 33
  13. Balling, L. W., Mølgaard, L. L., Townend, O., and Nielsen, J. B. B. (2021). The Collaboration between Hearing Aid Users and Artificial Intelligence to Optimize Sound. Seminars in Hearing. doi:10.1055/s-0041-1735135. Ch. 33
  14. Baltzell, L. S., Xia, J., and Kalluri, S. (2018). Efficient characterization of individual differences in compression ratio preference. The Journal of the Acoustical Society of America. doi:10.1121/1.5067390. Ch. 32 Ch. 33
  15. Basu, C., Yang, Q., Hungerman, D., Singhal, M., and Dragan, A. D. (2017). Do You Want Your Autonomous Car To Drive Like You? HRI 2017. Ch. 34
  16. Bemporad, A., and Piga, D. (2021). Global optimization based on active preference learning with radial basis functions. Machine Learning. Ch. 33
  17. Benavoli, A., Azzimonti, D., and Piga, D. (2021b). Choice functions based multi-objective Bayesian optimisation. arXiv. preprint Ch. 34
  18. Biswas, A., Liu, Y., Creange, N., Liu, Y.-C., Jesse, S., Yang, J.-C., … Vasudevan, R. K. (2024). A dynamic Bayesian optimized active recommender system for curiosity-driven partially Human-in-the-loop automated experiments. npj Computational Materials. Ch. 34
  19. Biswas, A., Funakubo, H., and Liu, Y. (2026). Human-AI Collaborative Autonomous Experimentation With Proxy Modeling for Comparative Observation. arXiv. preprint Ch. 34
  20. Bıyık, E., and Sadigh, D. (2018). Batch Active Preference-Based Learning of Reward Functions. CoRL 2018. Ch. 33
  21. Bıyık, E., Palan, M., Landolfi, N. C., Losey, D. P., and Sadigh, D. (2019). Asking Easy Questions: A User-Friendly Approach to Active Reward Learning. CoRL 2019. Ch. 33
  22. Bıyık, E., Huynh, N., Kochenderfer, M. J., and Sadigh, D. (2020). Active Preference-Based Gaussian Process Regression for Reward Learning. RSS 2020. Ch. 33
  23. Bıyık, E., Talati, A., and Sadigh, D. (2021). APReL: A Library for Active Preference-based Reward Learning Algorithms. arXiv. software Ch. 33
  24. Bıyık, E., Losey, D. P., Palan, M., Landolfi, N. C., Shevchuk, G., and Sadigh, D. (2022). Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferences. IJRR. Ch. 33
  25. Brochu, E., de Freitas, N., and Ghosh, A. (2007). Active Preference Learning with Discrete Choice Data. Advances in Neural Information Processing Systems. Ch. 32
  26. Broukhim, A., Shen, Y., Ammanabrolu, P., and Weibel, N. (2025). Preference-Based Learning in Audio Applications: A Systematic Analysis. arXiv. preprint Ch. 33
  27. Catkin, B., and Patoglu, V. (2023). Preference-Based Human-in-the-Loop Optimization for Perceived Realism of Haptic Rendering. IEEE Transactions on Haptics. doi:10.1109/toh.2023.3266726. Ch. 32
  28. Cella, C., Ristic, M., Faroni, M., Zanchettin, A. M., and Rocco, P. (2026). Adaptive Human-Robot Collaborative Painting Combining Preference-Based Optimization and Dynamic Motion Primitives. IEEE Robotics and Automation Letters. doi:10.1109/LRA.2026.3683596. Ch. 33
  29. Cercola, M., Lomuscio, M., Piga, D., and Formentin, S. (2026b). Regularized GLISp for sensor-guided human-in-the-loop optimization. IFAC Journal of Systems and Control. doi:10.1016/j.ifacsc.2026.100368. Ch. 34
  30. Chakraborty, T., Wirth, C., and Seifert, C. (2025a). Comparative Explanations: Explanation Guided Decision Making for Human-in-the-Loop Preference Selection. World Conference on eXplainable AI 2025. Ch. 32
  31. Chakraborty, T., Koelle, M., Schlötterer, J., Schlicker, N., Wirth, C., and Seifert, C. (2025b). Explanation format does not matter; but explanations do – An Eggsbert study on explaining Bayesian Optimisation tasks. Information Systems Frontiers. doi:10.1007/s10796-025-10671-6. Ch. 32
  32. Chan, L., Liao, Y.-C., Mo, G. B., Dudley, J. J., Cheng, C.-L., Kristensson, P. O., and Oulasvirta, A. (2022). Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques. CHI 2022. Ch. 32
  33. Chiu, C.-H., Koyama, Y., Lai, Y.-C., Igarashi, T., and Yue, Y. (2020). Human-in-the-loop differential subspace search in high-dimensional latent space. ACM Transactions on Graphics. doi:10.1145/3386569.3392409. Ch. 32
  34. Choi, D., Son, K., Yu, J., Jung, H., and Kim, J. (2026). IdeaBlocks: Expressing and Reusing Divergent Intents for Graphic Design Exploration using Generative AI. Proceedings of the 2026 Designing Interactive Systems Conference. doi:10.1145/3800645.3813005. Ch. 32
  35. Chong, T. L. H., Shen, I.-C., Sato, I., and Igarashi, T. (2021). Interactive Optimization of Generative Image Modelling using Sequential Subspace Search and Content-based Guidance. Computer Graphics Forum. doi:10.1111/cgf.14188. Ch. 32
  36. Choung, O.-H., Vianello, R., Segler, M., Stiefl, N., and Jiménez-Luna, J. (2023). Extracting medicinal chemistry intuition via preference machine learning. Nature Communications. doi:10.1038/s41467-023-42242-1. Ch. 34
  37. Colella, F., Daee, P., Jokinen, J., Oulasvirta, A., and Kaski, S. (2020). Human Strategic Steering Improves Performance of Interactive Optimization. UMAP 2020. Ch. 32
  38. Colley, M., Jansen, P., Keskar, M., and Rukzio, E. (2025). Improving External Communication of Automated Vehicles Using Bayesian Optimization. CHI 2025. Ch. 32
  39. Colley, M., Jansen, P., Krauss, S., and Rukzio, E. (2026). Multi-Session User Experience Assessments of Computationally Optimized Automated Vehicle Functionality Visualizations. Proceedings of the 18th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. doi:10.1145/3828157.3828785. Ch. 32
  40. Cosner, R., Tucker, M., Taylor, A., Li, K., Molnár, T., Ubelacker, W., … Ames, A. (2022). Safety-Aware Preference-Based Learning for Safety-Critical Control. Learning for Dynamics and Control Conference. Ch. 33
  41. Coutinho, J. P., Castillo, I., and Reis, M. S. (2024). Human-in-the-loop controller tuning using Preferential Bayesian Optimization. IFAC-PapersOnLine. doi:10.1016/j.ifacol.2024.08.306. Ch. 33
  42. Coutinho, J. P., Peng, Y., Rendall, R., Ma, K., Chin, S.-T., Castillo, I., and Reis, M. S. (2026). Efficient human-in-the-loop MPC tuning with multi-task preferential Bayesian optimization. Control Engineering Practice. Ch. 33
  43. Csomay-Shanklin, N., Tucker, M., Dai, M., Reher, J., and Ames, A. D. (2022). Learning Controller Gains on Bipedal Walking Robots via User Preferences. ICRA 2022. Ch. 33
  44. Dang, T., Pham, L.-H., Truong, S. T., Glenn, A., Nguyen, W., Pham, E. A., … Luong, T. (2025). Preferential Multi-Objective Bayesian Optimization for Drug Discovery. ICLR 2025 Workshop. workshop paper Ch. 34
  45. de Vries, W., van Kampen, J., and Salazar, M. (2024). A Human-optimized Model Predictive Control Scheme and Extremum Seeking Parameter Estimator for Slip Control of Electric Race Cars. arXiv. preprint Ch. 34
  46. De Witte, S., Taets, J., Retzler, A., Crevecoeur, G., and Lefebvre, T. (2025). How to Capture Human Preference: Commissioning of a Robotic Use-Case via Preferential Bayesian Optimisation. arXiv. preprint Ch. 33
  47. Deneault, J. R., Kim, W., Kim, J., Gu, Y., Chang, J., Maruyama, B., Myung, J. I., and Pitt, M. A. (2025). Preferential Bayesian optimization improves the efficiency of printing objects with subjective qualities. Digital Discovery. Ch. 34
  48. Díaz, M. A., Nuzzo, S., Flynn, L., Beckerle, P., Verstraten, T., and De Pauw, K. (2026). User preference in the personalized control of an ankle prosthesis: a case study. Journal of NeuroEngineering and Rehabilitation. doi:10.1186/s12984-026-01931-w. Ch. 33
  49. Ding, Y., Kim, M., Kuindersma, S., and Walsh, C. J. (2018). Human-in-the-Loop Optimization of Hip Assistance with a Soft Exosuit during Walking. Science Robotics. Ch. 33
  50. Dudley, J. J., Jacques, J. T., and Kristensson, P. O. (2019). Crowdsourcing Interface Feature Design with Bayesian Optimization. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3290605.3300482. Ch. 32
  51. Dudley, J., Oulasvirta, A., Chan, L., and Kristensson, P. O. (2026). Putting the Human Back in the Loop: A Review of Interactive Bayesian Optimization. ACM Computing Surveys. Ch. 32
  52. Farooqi, H., Zhao, Z., Darrow, D., Lamperski, A., and Netoff, T. I. (2025). An augmented preference-based Bayesian approach for optimizing neuromodulation stimulation parameters using meta learning. Journal of Neural Engineering. Ch. 33
  53. Fauvel, T., and Chalk, M. (2022). Human-in-the-loop optimization of visual prosthetic stimulation. Journal of Neural Engineering. Ch. 33
  54. Feith, N., and Rueckert, E. (2024). Integrating Human Expertise in Continuous Spaces: A Novel Interactive Bayesian Optimization Framework with Preference Expected Improvement. 2024 21st International Conference on Ubiquitous Robots (UR). doi:10.1109/ur61395.2024.10597501. Ch. 33
  55. Fernandes Junior, F. E., Langerak, T., Keränen, M., Shi, D., Alipova, A., and Oulasvirta, A. (2026). Integrating Multi-Source Feedback in Computational Design. Accepted at ACM TiiS. Ch. 32
  56. Fortuna, A., Lorenzini, M., Leonori, M., Gandarias, J., Balatti, P., Cho, Y., De Momi, E., and Ajoudani, A. (2024). A Personalizable Controller for the Walking Assistive omNi-Directional Exo-Robot (WANDER). ICRA 2024. Ch. 33
  57. Gilbert, M. L., Deroche, M. L. D., Jiradejvong, P., Chan Barrett, K., and Limb, C. J. (2022). Cochlear Implant Compression Optimization for Musical Sound Quality in MED-EL Users. Ear & Hearing. Ch. 33
  58. Gmeiner, F., Yang, H., Yao, L., Holstein, K., and Martelaro, N. (2023). Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design Tools. CHI 2023. Ch. 32
  59. González, J., Dai, Z., Damianou, A., and Lawrence, N. D. (2017). Preferential Bayesian Optimization. International Conference on Machine Learning. Ch. 32
  60. Granley, J., Fauvel, T., Chalk, M., and Beyeler, M. (2023). Human-in-the-Loop Optimization for Deep Stimulus Encoding in Visual Prostheses. NeurIPS 2023. Ch. 33
  61. Haltia, A., Hyvönen, V., and Kaski, S. (2026). Elicitation-Augmented Bayesian Optimization. arXiv. preprint Ch. 34
  62. Hawkins-Hooker, A., Duckworth, P., and Bent, O. (2023). Preferential Bayesian Optimisation for Protein Design with Ranking-Based Fitness Predictors. NeurIPS 2023 MLSB Workshop. workshop paper Ch. 34
  63. Hiranaka, A., Hwang, M., Lee, S., Wang, C., Fei-Fei, L., Wu, J., and Zhang, R. (2023). Primitive Skill-based Robot Learning from Human Evaluative Feedback. IROS 2023. Ch. 34
  64. Huber, F., Rojas Gonzalez, S., and Astudillo, R. (2025). Bayesian Preference Elicitation for Decision Support in Multi‐Objective Optimization. Journal of Multi-Criteria Decision Analysis. Ch. 34
  65. Ignatenko, T., Kondrashov, K., Cox, M., and de Vries, B. (2025). On preference learning based on sequential Bayesian optimization with pairwise comparison. Artificial Intelligence. Ch. 33
  66. Ingraham, K. A., Remy, C. D., and Rouse, E. J. (2022). The role of user preference in the customized control of robotic exoskeletons. Science Robotics. Ch. 33 Ch. 34
  67. Ingraham, K. A., Tucker, M., Ames, A. D., Rouse, E. J., and Shepherd, M. K. (2023). Leveraging user preference in the design and evaluation of lower-limb exoskeletons and prostheses. Current Opinion in Biomedical Engineering. Ch. 33
  68. Iwai, K., Kumagae, Y., Koyama, Y., Hamasaki, M., and Goto, M. (2025). Constrained Preferential Bayesian Optimization and Its Application in Banner Ad Design. Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Ch. 32
  69. Jansen, P. (2025). Human-in-the-Loop Optimization for Inclusive Design: Balancing Automation and Designer Expertise. CHI 2025 Workshop Access InContext. workshop paper Ch. 32
  70. Jansen, P., Colley, M., Krauß, S., Hirschle, D., and Rukzio, E. (2025). OptiCarVis: Improving Automated Vehicle Functionality Visualizations Using Bayesian Optimization to Enhance User Experience. CHI 2025. Ch. 32
  71. Janwani, N., Lerner, M. T., Young, A. J., and Tucker, M. (2026). Multi-Objective Human-in-the-Loop Bayesian Optimization of a Lower-Limb Exoskeleton. arXiv. preprint Ch. 33
  72. Kadner, F., Keller, Y., and Rothkopf, C. A. (2021). AdaptiFont: Increasing Individuals' Reading Speed with a Generative Font Model and Bayesian Optimization. CHI 2021. Ch. 32
  73. Kanarik, K. J., Osowiecki, W. T., Lu, Y., Talukder, D., Roschewsky, N., Park, S. N., … Gottscho, R. A. (2023). Human–machine collaboration for improving semiconductor process development. Nature. Ch. 34
  74. Kim, G., and Sergi, F. (2025). Validation of Dynamic Bayesian Optimization for a Non-Stationary Human-in-the-Loop Optimization Problem. bioRxiv. preprint Ch. 33
  75. Kim, E., Min, B., Xia, H., and Kim, J. (2026). Elicitive User Interfaces: Designing How Users Shape Generative Interfaces. arXiv. preprint Ch. 32
  76. Kobalczyk, K., Lin, Z. J., Letham, B., Zhao, Z., Balandat, M., and Bakshy, E. (2026). LILO: Bayesian Optimization with Natural Language Feedback. ICML 2026. Ch. 34
  77. Koch, J., Lucero, A., Hegemann, L., and Oulasvirta, A. (2019). May AI? Design Ideation with Cooperative Contextual Bandits. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3290605.3300863. Ch. 32
  78. Koyama, Y., and Goto, M. (2022). BO as Assistant: Using Bayesian Optimization for Asynchronously Generating Design Suggestions. Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. doi:10.1145/3526113.3545664. Ch. 32
  79. Koyama, Y., and Igarashi, T. (2018). Computational Design with Crowds. Computational Interaction. Ch. 32
  80. Koyama, Y., Sato, I., Sakamoto, D., and Igarashi, T. (2017). Sequential line search for efficient visual design optimization by crowds. ACM Transactions on Graphics. Ch. 32
  81. Koyama, Y., Sato, I., and Goto, M. (2020). Sequential Gallery for Interactive Visual Design Optimization. ACM Transactions on Graphics 39(4) (SIGGRAPH 2020). Ch. 32
  82. Kristiadi, A., Strieth-Kalthoff, F., Subramanian, S. G., Fortuin, V., Poupart, P., and Pleiss, G. (2024b). How Useful is Intermittent, Asynchronous Expert Feedback for Bayesian Optimization? AABI 2024. workshop paper Ch. 34
  83. Kuroki, S., Nakagawa, M., Yoshida, S., Koyama, Y., and Tadashi, K. (2026). LAPPI: Interactive Optimization with LLM-Assisted Preference-Based Problem Instantiation. IEEE Access 14. Ch. 32
  84. Kwon, Y., Tsurumine, Y., Shimmura, T., Kawamura, S., and Matsubara, T. (2022). Physically Consistent Preferential Bayesian Optimization for Food Arrangement. IEEE Robotics and Automation Letters. Ch. 33
  85. Langerak, T., Zhang, R., Wang, Z., Kristensson, P. O., and Oulasvirta, A. (2026). Cost-Aware Bayesian Optimization for Prototyping Interactive Devices. CHI 2026. Ch. 32
  86. Leahy, K., Daly, S. R., McKilligan, S., and Seifert, C. M. (2020). Design Fixation From Initial Examples: Provided Versus Self-Generated Ideas. Journal of Mechanical Design. Ch. 32
  87. Lee, U. H., Shetty, V. S., Franks, P. W., Tan, J., Evangelopoulos, G., Ha, S., and Rouse, E. J. (2023). User preference optimization for control of ankle exoskeletons using sample efficient active learning. Science Robotics. Ch. 33
  88. Lee, S. W., Choi, J., and Hyun, K. H. (2026). Part-level 3D shape generation driven by user intention inference with preferential Bayesian optimization. Scientific Reports. doi:10.1038/s41598-026-38916-7. Ch. 32
  89. Li, K., Tucker, M., Bıyık, E., Novoseller, E., Burdick, J. W., Sui, Y., … Ames, A. D. (2021). ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes. ICRA 2021. Ch. 33 Ch. 34
  90. Li, Z., Liao, Y.-C., and Holz, C. (2025a). Efficient Visual Appearance Optimization by Learning from Prior Preferences. UIST 2025. Ch. 32
  91. Li, Z., Gebhardt, C., Liao, Y.-C., and Holz, C. (2026a). Automating UI Optimization through Multi-Agentic Reasoning. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI 2026). doi:10.1145/3772318.3791444. Ch. 32
  92. Li, Y., Colley, M., Gui, X., Rendon Cardona, C. C., Jansen, P., Sandor, C., and Igarashi, T. (2026b). BlurDriving: Investigating How Personalized Blur Techniques Impact Drivers' Performance in Virtual Reality. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. doi:10.1145/3831646. Ch. 32
  93. Li, Z., Liao, Y.-C., and Holz, C. (2026f). Preference-Guided Prompt Optimization for Text-to-Image Generation. CHI 2026. Ch. 32
  94. Liao, Y.-C., Dudley, J. J., Mo, G. B., Cheng, C.-L., Chan, L., Oulasvirta, A., and Kristensson, P. O. (2023). Interaction Design With Multi-Objective Bayesian Optimization. IEEE Pervasive Computing. doi:10.1109/mprv.2022.3230597. Ch. 32
  95. Liao, Y.-C., Desai, R., Pierce, A. M., Taylor, K. E., Benko, H., Jonker, T. R., and Gupta, A. (2024a). A Meta-Bayesian Approach for Rapid Online Parametric Optimization for Wrist-based Interactions. Proceedings of the CHI Conference on Human Factors in Computing Systems. doi:10.1145/3613904.3642071. Ch. 32
  96. Liao, Y.-C., Mo, G. B., Dudley, J. J., Cheng, C.-L., Chan, L., Kristensson, P. O., and Oulasvirta, A. (2024b). Practical approaches to group-level multi-objective Bayesian optimization in interaction technique design. Collective Intelligence. doi:10.1177/26339137241241313. Ch. 32
  97. Liao, Y.-C., Streli, P., Li, Z., Gebhardt, C., and Holz, C. (2025). Continual Human-in-the-Loop Optimization. CHI 2025. Ch. 32
  98. Liao, Y.-C., Belo, J., Moon, H.-S., Steimle, J., and Feit, A. M. (2026). Efficient Human-in-the-Loop Optimization via Priors Learned from User Models. CHI 2026. Ch. 32
  99. Lin, Z. J., Astudillo, R., Frazier, P., and Bakshy, E. (2022). Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes. International Conference on Artificial Intelligence and Statistics. Ch. 34
  100. Liu, Y., and Kalinin, S. V. (2025). Pareto-Optimal Experimentation: Human-Guided Multi-Objective Bayesian Optimization in Scanning Probe Microscopy. Nano Letters. Ch. 34
  101. Liu, C., Ling, S., and Jacobson, A. (2026b). GimmBO: Interactive Generative Image Model Merging via Bayesian Optimization. ACM Transactions on Graphics. doi:10.1145/3811293. Ch. 32
  102. Liu, X.-Y., Li, G., Wang, W., and Hou, Z.-G. (2026d). Personalized Lower-limb Exoskeleton Assistance via Preference-based Bayesian Optimization. arXiv. preprint Ch. 33
  103. Maberry, A., and Martin, A. E. (2026). Just Noticeable Difference of Impedance Parameters While Walking in an Ankle Exoskeleton. IEEE Transactions on Neural Systems and Rehabilitation Engineering. Ch. 33
  104. Marcos, M., Mur-Labadia, L., and Martinez-Cantin, R. (2025). Random rotational embedding Bayesian optimization for human-in-the-loop personalized music generation. PLOS One. doi:10.1371/journal.pone.0335853. Ch. 32
  105. McCourt, M., and Dewancker, I. (2019). Sampling Humans for Optimizing Preferences in Coloring Artwork. ICML 2019 Workshop on Human in the Loop Learning. workshop paper Ch. 32
  106. Meinhardt, L.-M., Schramm, C., Jansen, P., Colley, M., and Rukzio, E. (2025). Fly Away: Evaluating the Impact of Motion Fidelity on Optimized User Interface Design via Bayesian Optimization in Automated Urban Air Mobility Simulations. CHI 2025. Ch. 32
  107. Menn, J., Stenger, D., and Trimpe, S. (2026b). Preferential Bayesian Optimization with Crash Feedback. IEEE Robotics and Automation Letters. doi:10.1109/LRA.2026.3665446. Ch. 33
  108. Meta Platforms, Inc. (2026d). Bayesian optimization with preference exploration (BOPE tutorial, documentation v0.18.1). botorch.org. software Ch. 34
  109. Meta Platforms, Inc. (2026l). CHANGELOG (versions 1.2 to 1.3). GitHub. software Ch. 34
  110. Mikkola, P., Todorović, M., Järvi, J., Rinke, P., and Kaski, S. (2020). Projective Preferential Bayesian Optimization. International Conference on Machine Learning. Ch. 32 Ch. 34
  111. Mo, G., Dudley, J., Chan, L., Liao, Y.-C., Oulasvirta, A., and Kristensson, P. O. (2024). Cooperative Multi-Objective Bayesian Design Optimization. ACM Transactions on Interactive Intelligent Systems. doi:10.1145/3657643. Ch. 32
  112. Myers, V., Bıyık, E., Anari, N., and Sadigh, D. (2021). Learning Multimodal Rewards from Rankings. CoRL 2021. Ch. 33
  113. Nandy, A., and Goucher-Lambert, K. (2025). Exploring the Effectiveness of Interactive Preference Learning for Adapting Designs to Abstract Semantic Attributes. Journal of Mechanical Design. Ch. 32
  114. Nielsen, J., Nielsen, J., and Larsen, J. (2015). Perception-based Personalization of Hearing Aids using Gaussian Processes and Active Learning. IEEE/ACM Transactions on Audio, Speech, and Language Processing. Ch. 33
  115. Niwa, R., Yoshida, S., Koyama, Y., and Ushiku, Y. (2025). Cooperative Design Optimization through Natural Language Interaction. UIST 2025. Ch. 32
  116. Ou, C., Buschek, D., Mayer, S., and Butz, A. (2022). The Human in the Infinite Loop: A Case Study on Revealing and Explaining Human-AI Interaction Loop Failures. Mensch und Computer 2022. Ch. 32 Ch. 34
  117. Ou, C., Mayer, S., and Butz, A. (2023). The Impact of Expertise in the Loop for Exploring Machine Rationality. IUI 2023. Ch. 32
  118. Owaki, T., Koyama, Y., Nakano, T., Yamaguchi, T., Goto, M., and Sakai, H. (2026). Learning Feasibility-Aware Latent Spaces for Preference-Based Exploration of Procedural Automotive Wheel Designs. arXiv. preprint Ch. 32
  119. Palan, M., Landolfi, N. C., Shevchuk, G., and Sadigh, D. (2019). Learning Reward Functions by Integrating Human Demonstrations and Preferences. RSS 2019. Ch. 33
  120. Park, K., and Collins, S. H. (2026). Simultaneous Forward and Inverse Human-in-the-Loop Optimization. CoRL 2026. Ch. 33
  121. Peng, Y.-H., Bigham, J. P., and Wu, J. (2026). Efficient Personalization of Generative User Interfaces. arXiv. preprint Ch. 32
  122. Poggensee, K. L., and Collins, S. H. (2021). How adaptation, training, and customization contribute to benefits from exoskeleton assistance. Science Robotics. Ch. 33
  123. Previtali, D., Mazzoleni, M., Ferramosca, A., and Previdi, F. (2023). GLISp-r: a preference-based optimization algorithm with convergence guarantees. Computational Optimization and Applications. Ch. 33
  124. Rajagopalan, R., Dutta, D., Wei, Y.-L., and Roy Choudhury, R. (2026). Personalized Image Generation via Human-in-the-loop Bayesian Optimization. International Conference on Machine Learning. Ch. 32
  125. Ramella, G., Ijspeert, A., and Bouri, M. (2025). Rapid Online Learning of Hip Exoskeleton Assistance Preferences. ICRA 2025. Ch. 33
  126. Ran, W., Chen, H., Xia, T., Nishimura, Y., Guo, C., and Yin, Y. (2023). Online Personalized Preference Learning Method Based on In-Formative Query for Lane Centering Control Trajectory. Sensors. doi:10.3390/s23115246. Ch. 34
  127. Rodemann, J., Croppi, F., Arens, P., Sale, Y., Herbinger, J., Bischl, B., … Casalicchio, G. (2024). Explaining Bayesian Optimization by Shapley Values Facilitates Human-AI Collaboration. arXiv. preprint Ch. 32
  128. Roveda, L., Maggioni, B., Marescotti, E., Shahid, A. A., Maria Zanchettin, A., Bemporad, A., and Piga, D. (2021). Pairwise Preferences-Based Optimization of a Path-Based Velocity Planner in Robotic Sealing Tasks. IEEE Robotics and Automation Letters. Ch. 33
  129. Sadigh, D., Dragan, A., Sastry, S., and Seshia, S. (2017). Active Preference-Based Learning of Reward Functions. Robotics: Science and Systems XIII. Ch. 33
  130. Sandholtz, N., Miyamoto, Y., Bornn, L., and Smith, M. (2023). Inverse Bayesian Optimization: Learning Human Acquisition Functions in an Exploration vs Exploitation Search Task. Bayesian Analysis. doi:10.1214/21-BA1303. Ch. 32
  131. Sankagiri, S., Etesami, J., Fatemi, P., and Grossglauser, M. (2026). Recycling History: Efficient Recommendations from Contextual Dueling Bandits. Algorithmic Learning Theory. Ch. 34
  132. Saracay, I., Schmidt, L., and Guestrin, C. (2026). Beyond expert users: agents should help users construct preferences, not just elicit them. Conference on Language Modeling (COLM 2026). Ch. 32
  133. Schäfer, N., Zhao, G., Li, B., Kupnik, M., Seyfarth, A., Beckerle, P., and Grimmer, M. (2026). User preference-based human-in-the-loop tuning of exoskeleton assistance during walking. npj Biomedical Innovations. doi:10.1038/s44385-026-00085-7. Ch. 33
  134. Schoinas, E., Rastogi, A., Carter, A., Granley, J., and Beyeler, M. (2025). Evaluating Deep Human-in-the-Loop Optimization for Retinal Implants Using Sighted Participants. 2025 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). doi:10.1109/embc58623.2025.11253762. Ch. 32 Ch. 33
  135. Shao, K., Chakrabarty, A., Mesbah, A., and Romeres, D. (2024). Coactive Preference-Guided Multi-Objective Bayesian Optimization: An Application to Policy Learning in Personalized Plasma Medicine. IEEE Control Systems Letters. Ch. 33
  136. Shen, J., Hu, J., Dudley, J. J., and Kristensson, P. O. (2022). Personalization of a Mid-Air Gesture Keyboard using Multi-Objective Bayesian Optimization. 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). doi:10.1109/ismar55827.2022.00088. Ch. 32
  137. Slade, P., Kochenderfer, M. J., Delp, S. L., and Collins, S. H. (2022). Personalizing exoskeleton assistance while walking in the real world. Nature. Ch. 33
  138. Slade, P., Atkeson, C., Donelan, J. M., Houdijk, H., Ingraham, K. A., Kim, M., … Collins, S. H. (2024). On human-in-the-loop optimization of human–robot interaction. Nature. Ch. 33
  139. Søgaard Jensen, N., Hau, O., Bagger Nielsen, J. B., Bundgaard Nielsen, T., and Vase Legarth, S. (2019). Perceptual Effects of Adjusting Hearing-Aid Gain by Means of a Machine-Learning Approach Based on Individual User Preference. Trends in Hearing. doi:10.1177/2331216519847413. Ch. 32 Ch. 33 Ch. 34
  140. Song, Y., Gebhardt, C., Liao, Y.-C., and Holz, C. (2025). Preference-Guided Multi-Objective UI Adaptation. Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology. doi:10.1145/3746059.3747645. Ch. 32
  141. Sui, Y., Yue, Y., and Burdick, J. W. (2017a). Correlational Dueling Bandits with Application to Clinical Treatment in Large Decision Spaces. IJCAI 2017. Ch. 33
  142. Sui, Y., Zhuang, V., Burdick, J., and Yue, Y. (2018b). Stagewise Safe Bayesian Optimization with Gaussian Processes. International Conference on Machine Learning. Ch. 32 Ch. 33
  143. Sun, L., Ma, H., An, H., and Wei, Q. (2024). An Individual Prosthesis Control Method with Human Subjective Choices. Biomimetics. doi:10.3390/biomimetics9020077. Ch. 33
  144. Sundin, I., Voronov, A., Xiao, H., Papadopoulos, K., Bjerrum, E. J., Heinonen, M., … Engkvist, O. (2022). Human-in-the-loop assisted de novo molecular design. Journal of Cheminformatics. doi:10.1186/s13321-022-00667-8. Ch. 34
  145. Susak, J., Liu, Y., Jansen, P., and Colley, M. (2026). ProVoice: Designing Proactive Functionality for In-Vehicle Conversational Assistants using Multi-Objective Bayesian Optimization to Enhance Driver Experience. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. doi:10.1145/3772318.3791877. Ch. 32
  146. Taddei, S., Koppen, W., Alfio, E., Nuzzo, S., Flynn, L., Diaz, M. A., … Verstraten, T. (2026). Bayesian Preference Elicitation: Human-In-The-Loop Optimization of An Active Prosthesis. arXiv. preprint Ch. 33
  147. Tanaka, M., Ono, K., and Yamada, S. (2026). Human-in-the-Loop Bayesian Optimization Approach to Supporting Early-Stage Architectural Design. CAADRIA proceedings. doi:10.52842/conf.caadria.2026.1.347. Ch. 32
  148. Tasnim, N. Z., Ni, A., Lobarinas, E., and Kehtarnavaz, N. (2024). A Review of Machine Learning Approaches for the Personalization of Amplification in Hearing Aids. Sensors. doi:10.3390/s24051546. Ch. 33
  149. Tatsukawa, Y., Shen, I.-C., Dogan, M. D., Qi, A., Koyama, Y., Shamir, A., and Igarashi, T. (2025). FontCraft: Multimodal Font Design Using Interactive Bayesian Optimization. CHI 2025. Ch. 32
  150. Theiner, L., Hirt, S., Steinke, A., and Findeisen, R. (2025). Exploiting Prior Knowledge in Preferential Learning of Individualized Autonomous Vehicle Driving Styles. ECC 2025. Ch. 34
  151. Theiner, L., Pfefferkorn, M., Zhao, Y., Hirt, S., and Findeisen, R. (2026). Efficient Controller Learning from Human Preferences and Numerical Data Via Multi-Modal Surrogate Models. European Control Conference. Ch. 34
  152. Tucker, M. (2023). Enabling Robust and User-Customized Bipedal Locomotion on Lower-Body Assistive Devices via Hybrid System Theory and Preference-Based Learning. California Institute of Technology. doi:10.7907/j9hk-xa17. thesis Ch. 33
  153. Tucker, M., Cheng, M., Novoseller, E., Cheng, R., Yue, Y., Burdick, J. W., and Ames, A. D. (2020a). Human Preference-Based Learning for High-dimensional Optimization of Exoskeleton Walking Gaits. IROS 2020. Ch. 33
  154. Tucker, M., Novoseller, E., Kann, C., Sui, Y., Yue, Y., Burdick, J. W., and Ames, A. D. (2020b). Preference-Based Learning for Exoskeleton Gait Optimization. 2020 IEEE International Conference on Robotics and Automation (ICRA). Ch. 33
  155. Tucker, M., Csomay-Shanklin, N., Ma, W.-L., and Ames, A. D. (2021). Preference-Based Learning for User-Guided HZD Gait Generation on Bipedal Walking Robots. ICRA 2021. Ch. 33
  156. Tucker, M., Li, K., Yue, Y., and Ames, A. D. (2022). POLAR: Preference Optimization and Learning Algorithms for Robotics. arXiv. preprint Ch. 33
  157. Vyas, D., Brummet, R., Anwar, Y., Jensen, J., Jorgensen, E., Wu, Y.-H., and Chipara, O. (2022). Personalizing over-the-counter hearing aids using pairwise comparisons. Smart Health. doi:10.1016/j.smhl.2021.100231. Ch. 33 Ch. 34
  158. Wadinambiarachchi, S., Kelly, R. M., Pareek, S., Zhou, Q., and Velloso, E. (2024). The Effects of Generative AI on Design Fixation and Divergent Thinking. Proceedings of the CHI Conference on Human Factors in Computing Systems. Ch. 32
  159. Wang, W., Xu, W., and Jones, C. N. (2025c). Human-in-the-loop: Real-time Preference Optimization. arXiv. preprint Ch. 34
  160. Wang, W., Shi, J., and Jones, C. N. (2025d). Personalized Building Climate Control with Contextual Preferential Bayesian Optimization. arXiv. preprint Ch. 34
  161. Weichert, D., Ernis, G., Worthmann, M., Ryzko, P., and Seifert, L. (2025). When Less is More: A Story of Failing Bayesian Optimization Due to Additional Expert Knowledge. arXiv. preprint Ch. 32 Ch. 34
  162. Wen, C., Phung, T., Mehrotra, P., Gulwani, S., Beaty, R. E., Nagashima, T., and Singla, A. (2026). Exploration vs. Fixation: Scaffolding Divergent and Convergent Thinking for Human-AI Co-Creation with Generative Models. arXiv. preprint Ch. 32
  163. Xiong, J., Lee, S., Karava, P., and Tzempelikos, A. (2017). Personalized visual satisfaction profiles from comparative preferences using Bayesian inference. Energy Procedia. Ch. 34
  164. Xiong, J., Tzempelikos, A., Bilionis, I., Awalgaonkar, N. M., Lee, S., Konstantzos, I., Sadeghi, S. A., and Karava, P. (2018). Inferring personalized visual satisfaction profiles in daylit offices from comparative preferences using a Bayesian approach. Building and Environment. Ch. 34
  165. Xiong, J., Awalgaonkar, N. M., Tzempelikos, A., Bilionis, I., and Karava, P. (2020). Efficient learning of personalized visual preferences in daylit offices: An online elicitation framework. Building and Environment. Ch. 34
  166. Xu, W., Wang, W., Jiang, Y., Svetozarevic, B., and Jones, C. (2024b). Principled Preferential Bayesian Optimization. International Conference on Machine Learning. Ch. 34
  167. Xu, P., Zheng, S., Ye, Y., Bai, C., Xu, S., Geng, H., Ho, T.-Y., and Yu, B. (2026). RankTuner: When Design Tool Parameter Tuning Meets Preference Bayesian Optimization. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems. Ch. 34
  168. Yamamoto, K., Koyama, Y., and Ochiai, Y. (2022). Photographic Lighting Design with Photographer-in-the-Loop Bayesian Optimization. Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. doi:10.1145/3526113.3545690. Ch. 32
  169. Yuan, L.-P., Dudley, J. J., Kristensson, P. O., and Qu, H. (2025). Personalized Dual-Level Color Grading for 360-degree Images in Virtual Reality. IEEE Transactions on Visualization and Computer Graphics. Ch. 32
  170. Zhang, J., Fiers, P., Witte, K. A., Jackson, R. W., Poggensee, K. L., Atkeson, C. G., and Collins, S. H. (2017). Human-in-the-loop optimization of exoskeleton assistance during walking. Science. Ch. 33
  171. Zhang, R., Zhu, X., Pourebadi Khotbehsara, M., Dao, W., Bıyık, E., and Culbertson, H. (2026b). Vibrotactile Preference Learning: Uncertainty-Aware Preference Learning for Personalized Vibration Feedback. UMAP 2026 (per Semantic Scholar). Ch. 32
  172. Zhao, Z., Ahmadi, A., Hoover, C., Grado, L., Peterson, N., Wang, X., … Netoff, T. I. (2021). Optimization of Spinal Cord Stimulation Using Bayesian Preference Learning and Its Validation. IEEE Transactions on Neural Systems and Rehabilitation Engineering. doi:10.1109/tnsre.2021.3113636. Ch. 33
  173. Zhou, Y., Koyama, Y., Goto, M., and Igarashi, T. (2020). Generative Melody Composition with Human-in-the-Loop Bayesian Optimization. CSMC-MuMe 2020. Ch. 32
  174. Zhou, Y., Koyama, Y., Goto, M., and Igarashi, T. (2021). Interactive Exploration-Exploitation Balancing for Generative Melody Composition. 26th International Conference on Intelligent User Interfaces. Ch. 32
  175. Zhu, M., Bemporad, A., and Piga, D. (2021). Preference-based MPC calibration. ECC 2021. Ch. 34
  176. Zhu, M., Piga, D., and Bemporad, A. (2022). C-GLISp: Preference-Based Global Optimization Under Unknown Constraints With Applications to Controller Calibration. IEEE Transactions on Control Systems Technology. Ch. 32 Ch. 34