Synthesis
The book closes by drawing its threads together. A single comparison turns out to mix several things: a stable preference, structured noise in how options are evaluated, change caused by the question itself, and answers that are incomplete or deliberately random. That makes preferential optimization both an estimate and an intervention, and it changes what a good system and a good study look like.
The three chapters say what a comparison measures, give concrete recommendations for building and evaluating a preferential optimization system (including when a simpler method should win), and end with the open problems, the one experiment that would most change our understanding, and an outlook on the research directions where the problem these methods address is growing.
The part draws on the whole book but is written to be readable on its own.
Chapters in this part
- 45 What a Comparison Measures
Where the field's difficulty now lies, which results from other disciplines survived scrutiny, and what one comparison measures: a stable preference, structured evaluation noise, change caused by the query, and answers with no preference behind them. A preferential optimization session is both an estimate and an intervention.
- 46 Building and Evaluating a Preferential Optimization System
Concrete recommendations for deciding whether to use preferential optimization at all, and then for the observation model, the surrogate, query design, non-stationarity, stopping, evaluation, and ethics, with the situations in which a simpler method should be the default.
- 47 Open Problems and the Decisive Experiment
Eighteen open problems, each with an experiment that would settle it, the one experiment nobody has run (randomize the query order and retest a week later) described concretely enough to run, and the conclusion of the book.
References for Part X
127 works cited across this part's chapters.
- (2026). Benchmarking self-driving labs. Digital Discovery. Ch. 46 Ch. 47
- (2025). Ranges of Randomization. Review of Economics and Statistics. Ch. 45
- (2023). Identifying Nontransitive Preferences. University of Zurich. working paper Ch. 45
- (2022). Evaluating Deliberative Competence: A Simple Method with an Application to Financial Choice. American Economic Review. Ch. 45
- (2025). No evidence for decision fatigue using large-scale field data from healthcare. Communications Psychology. Ch. 45
- (2022). Solutions to preference manipulation in recommender systems require knowledge of meta-preferences. FAccTRec Workshop (RecSys 2022). workshop paper Ch. 45
- (2023). qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimization. International Conference on Artificial Intelligence and Statistics. Ch. 45
- (2025). A systematic review and meta-analyses of the temporal stability and convergent validity of risk preference measures. Nature Human Behaviour. doi:10.1038/s41562-024-02085-2. Ch. 45
- (2023). The functional form of value normalization in human reinforcement learning. eLife. Ch. 45
- (2018). Reference-point centering and range-adaptation enhance human reinforcement learning at the cost of irrational preferences. Nature Communications. Ch. 45
- (2024). The online metacognitive control of decisions. Communications Psychology. doi:10.1038/s44271-024-00071-y. Ch. 46
- (2009). Beyond Revealed Preference: Choice-Theoretic Foundations for Behavioral Welfare Economics *. Quarterly Journal of Economics. Ch. 45
- (2017). Noisy preferences in risky choice: A cautionary note. Psychological Review. Ch. 45
- (2022). A meta-analysis on the effect of visual attention on choice. Journal of Experimental Psychology: General. Ch. 46
- (2019). Asking Easy Questions: A User-Friendly Approach to Active Reward Learning. CoRL 2019. Ch. 46
- (2024). Modelling individual aesthetic judgements over time. Philosophical Transactions of the Royal Society B: Biological Sciences. Ch. 45
- (2024). Meta-analysis of Empirical Estimates of Loss Aversion. Journal of Economic Literature. Ch. 45 Ch. 46
- (2022). Estimating and Penalizing Induced Preference Shifts in Recommender Systems. ICML 2022. Ch. 45 Ch. 46
- (2023). Characterizing Manipulation from AI Systems. Equity and Access in Algorithms, Mechanisms, and Optimization. Ch. 45 Ch. 46
- (2024). AI Alignment with Changing and Influenceable Reward Functions. International Conference on Machine Learning. Ch. 45 Ch. 47
- (2019). Revealed preferences under uncertainty: Incomplete preferences and preferences for randomization. Journal of Economic Theory. Ch. 45
- (2024). What’s so Hard about Hard Choices? Erasmus Journal for Philosophy and Economics. doi:10.23941/ejpe.v17i1.872. Ch. 45
- (2026). Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences. International Conference on Artificial Intelligence and Statistics. Ch. 45 Ch. 46
- (2023). Extracting medicinal chemistry intuition via preference machine learning. Nature Communications. doi:10.1038/s41467-023-42242-1. Ch. 45
- (2022). Choice, deferral, and consistency. Quantitative Economics. Ch. 45
- (2025). How to Capture Human Preference: Commissioning of a Robotic Use-Case via Preferential Bayesian Optimisation. arXiv. preprint Ch. 45
- (2022). Preference Dynamics Under Personalized Recommendations. EC 2022. Ch. 45 Ch. 46 Ch. 47
- (2025). Preferential Bayesian optimization improves the efficiency of printing objects with subjective qualities. Digital Discovery. Ch. 47
- (2026). User preference in the personalized control of an ankle prosthesis: a case study. Journal of NeuroEngineering and Rehabilitation. doi:10.1186/s12984-026-01931-w. Ch. 46
- (2018). Human-in-the-Loop Optimization of Hip Assistance with a Soft Exosuit during Walking. Science Robotics. Ch. 47
- (2024). Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimization. ICML 2024. Ch. 46
- (2026). We Still Don't Understand High-Dimensional Bayesian Optimization. AISTATS 2026 (best student paper). Ch. 45 Ch. 46
- (2010). Reconsidering the Effect of Market Experience on the "Endowment Effect". Econometrica. Ch. 45
- (2021). Choice changes preferences, not merely reflects them: A meta-analysis of the artifact-free free-choice paradigm. Journal of Personality and Social Psychology. Ch. 45 Ch. 47
- (2025). Consecutive Preferential Bayesian Optimization. arXiv. preprint Ch. 46
- (2022b). Regulation (EU) 2022/2065 on a Single Market for Digital Services (Digital Services Act). Official Journal of the European Union (EUR-Lex). non-peer-reviewed Ch. 46
- (2011). On the multi-utility representation of preference relations. Journal of Mathematical Economics. Ch. 45
- (2021). Human-in-the-loop optimization of retinal prostheses encoders. Sorbonne Université. thesis Ch. 47
- (2026). Agents, Alignment, and the Many Faces of Autonomy. Minds and Machines. Ch. 45
- (2020). Discrete Choice and Rational Inattention: A General Equivalence Result. International Economic Review. Ch. 45
- (2021). Preference stability in discrete choice experiments. Some evidence using eye-tracking. Journal of Behavioral and Experimental Economics. Ch. 45
- (2014). The Limits of Attraction. Journal of Marketing Research. Ch. 45
- (2019). Goal congruency dominates reward value in accounting for behavioral and neural correlates of value-based decision-making. Nature Communications. Ch. 45
- (2024). Axioms for AI Alignment from Human Feedback. Advances in Neural Information Processing Systems 37. Ch. 45
- (2023). The Effect of Modeling Human Rationality Level on Learning Rewards from Multiple Feedback Types. AAAI. Ch. 45 Ch. 46
- (2025). Optimal adaptive Bayesian design in choice experiments: performance for the population versus individuals. Marketing Letters. Ch. 46
- (2025). How human–AI feedback loops alter human perceptual, emotional and social judgements. Nature Human Behaviour. doi:10.1038/s41562-024-02077-2. Ch. 45 Ch. 46
- (2026). Multiobjective Optimisation for Others: How Anchoring Effects Change Based on Who Guides the Interaction. Journal of Multi-Criteria Decision Analysis. Ch. 46
- (2025). Performative Prediction: Past and Future. Statistical Science. Ch. 45
- (2005). The Impact of Utility Balance and Endogeneity in Conjoint Analysis. Marketing Science. Ch. 46
- (2019). Active ranking from pairwise comparisons and when parametric assumptions do not help. The Annals of Statistics. Ch. 45
- (2019). Graph Resistance and Learning from Pairwise Comparisons. ICML. Ch. 45 Ch. 46
- (2024). Vanilla Bayesian Optimization Performs Great in High Dimensions. International Conference on Machine Learning. Ch. 46 Ch. 47
- (2022). The role of user preference in the customized control of robotic exoskeletons. Science Robotics. Ch. 46
- (2011). Statistical ranking and combinatorial Hodge theory. Mathematical Programming. Ch. 45
- (2017). Contemporary Guidance for Stated Preference Studies. Journal of the Association of Environmental and Resource Economists. Ch. 46
- (2026). Constructive Alignment: Governing Preference Dynamics in Human-AI Interaction. AAAI-26 Workshop on Machine Ethics. workshop paper Ch. 45 Ch. 46
- (2025). Bayesian Optimization from Human Feedback: Near-Optimal Regret Bounds. International Conference on Machine Learning. Ch. 45 Ch. 47
- (2024). Stochastic Monotonicity and Random Utility Models: The Good and The Ugly. arXiv preprint 2409.00704. preprint Ch. 46
- (2024). On the Pros and Cons of Active Learning for Moral Preference Elicitation. AIES. Ch. 46
- (2026). Moral Change or Noise? On Problems of Aligning AI With Temporally Unstable Human Feedback. AAAI. Ch. 45 Ch. 46 Ch. 47
- (2026). PRISM-X: Experiments on Personalised Fine-Tuning with Human and Simulated Users. arXiv. preprint Ch. 46 Ch. 47
- (2020). Sequential Gallery for Interactive Visual Design Optimization. ACM Transactions on Graphics 39(4) (SIGGRAPH 2020). Ch. 46
- (2026). A Finite Time Analysis of Thompson Sampling for Bayesian Optimization with Preferential Feedback. International Conference on Artificial Intelligence and Statistics. Ch. 45 Ch. 47
- (2024a). Enhancing Preference-based Linear Bandits via Human Response Time. Advances in Neural Information Processing Systems. Ch. 45
- (2026c). Curiosity is Knowledge: Self-Consistent Learning and No-Regret Optimization with Active Inference. arXiv preprint 2602.06029. preprint Ch. 45
- (2026f). Preference-Guided Prompt Optimization for Text-to-Image Generation. CHI 2026. Ch. 47
- (2026). Efficient Human-in-the-Loop Optimization via Priors Learned from User Models. CHI 2026. Ch. 46
- (2026b). GimmBO: Interactive Generative Image Model Merging via Bayesian Optimization. ACM Transactions on Graphics. doi:10.1145/3811293. Ch. 46 Ch. 47
- (2020). Testing the Random Utility Hypothesis Directly. The Economic Journal. doi:10.1093/ej/uez039. Ch. 45
- (2010). NAUTILUS method: An interactive technique in multiobjective optimization based on the nadir point. European Journal of Operational Research. Ch. 46
- (2021). Whence the Expected Free Energy? Neural Computation. Ch. 45
- (2022). When Choices Are Mistakes. American Economic Review. Ch. 45 Ch. 46
- (2026). Revealed Incomplete Preferences. Working paper (author's website). working paper Ch. 45 Ch. 46 Ch. 47
- (2015). On making the right choice: A meta-analysis and large-scale replication attempt of the unconscious thought advantage. Judgment and Decision Making. Ch. 45
- (2025). Cooperative Design Optimization through Natural Language Interaction. UIST 2025. Ch. 45 Ch. 46
- (2017). The evolution of paired preference tests from forced choice to the use of ‘No Preference’ options, from preference frequencies to d′ values, from placebo pairs to signal detection. Trends in Food Science & Technology. Ch. 46 Ch. 47
- (2026b). Random Is Hard to Beat: Active Selection in online DPO with Modern LLMs. ICLR 2026 Workshop: I Can't Believe It's Not Better (ICBINB). workshop paper Ch. 45 Ch. 46 Ch. 47
- (2022). Indifference, indecisiveness, experimentation, and stochastic choice. Theoretical Economics. Ch. 45 Ch. 46 Ch. 47
- (2024). Decisions under Risk Are Decisions under Complexity. American Economic Review. Ch. 45
- (2022). The Human in the Infinite Loop: A Case Study on Revealing and Explaining Human-AI Interaction Loop Failures. Mensch und Computer 2022. Ch. 45 Ch. 46
- (2026). Learning Feasibility-Aware Latent Spaces for Preference-Based Exploration of Procedural Automotive Wheel Designs. arXiv. preprint Ch. 46
- (2024). Bandits with Preference Feedback: A Stackelberg Game Perspective. Advances in Neural Information Processing Systems. doi:10.52202/079017-0383. Ch. 45
- (2026). Efficient Personalization of Generative User Interfaces. arXiv. preprint Ch. 45 Ch. 46
- (2020). Performative Prediction. ICML. Ch. 45
- (2023). Nudging for changing selves. Synthese. Ch. 45 Ch. 46
- (2021). How adaptation, training, and customization contribute to benefits from exoskeleton assistance. Science Robotics. Ch. 46 Ch. 47
- (2019). Efficient coding of subjective value. Nature Neuroscience. Ch. 45
- (2023). The intrinsic variance of beauty judgment. Attention, Perception, & Psychophysics. Ch. 45
- (2022). Efficient coding of numbers explains decision bias and noise. Nature Human Behaviour. Ch. 45
- (2026). Personalized Image Generation via Human-in-the-loop Bayesian Optimization. International Conference on Machine Learning. Ch. 47
- (2026). Observation Moderates the Moral Licensing Effect: A Meta-Analytic Test of Interpersonal and Intrapsychic Mechanisms. Personality and Social Psychology Bulletin. Ch. 45
- (2019). Ellipsoidal Methods for Adaptive Choice-Based Conjoint Analysis. Operations Research. Ch. 46
- (2025). Preference Learning with Response Time: Robust Losses and Guarantees. NeurIPS. Ch. 46
- (2026). User preference-based human-in-the-loop tuning of exoskeleton assistance during walking. npj Biomedical Innovations. doi:10.1038/s44385-026-00085-7. Ch. 45 Ch. 46 Ch. 47
- (2025). Evaluating Deep Human-in-the-Loop Optimization for Retinal Implants Using Sighted Participants. 2025 47th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). doi:10.1109/embc58623.2025.11253762. Ch. 45 Ch. 46
- (2016). Estimation from Pairwise Comparisons: Sharp Minimax Bounds with Topology Dependence. Journal of Machine Learning Research. Ch. 45 Ch. 46
- (2026). Adaptive KappaSharp: Condition-Number Shaping for Preferential Bayesian Optimization. arXiv. preprint Ch. 45 Ch. 46 Ch. 47
- (2025b). Early versus late noise differentially enhances or degrades context-dependent choice. Nature Communications. doi:10.1038/s41467-025-59140-3. Ch. 45 Ch. 46
- (2022). High-value decisions are fast and accurate, inconsistent with diminishing value sensitivity. Proceedings of the National Academy of Sciences. Ch. 45 Ch. 46
- (2021). Bayesian reaction optimization as a tool for chemical synthesis. Nature. Ch. 47
- (2024). Response Time Improves Gaussian Process Models for Perception and Preferences. Uncertainty in Artificial Intelligence. Ch. 45 Ch. 46 Ch. 47
- (2021). Preferential Batch Bayesian Optimization. IEEE MLSP 2021. Ch. 46
- (2024). Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF. ICLR 2024. Ch. 45
- (2024). Behavioral Biases Are Temporally Stable. Working paper (author's website). working paper Ch. 45
- (2022). The Network HHD: Quantifying Cyclic Competition in Trait-Performance Models of Tournaments. SIAM Review. Ch. 45
- (2018b). Stagewise Safe Bayesian Optimization with Gaussian Processes. International Conference on Machine Learning. Ch. 47
- (2026). Bayesian Preference Elicitation: Human-In-The-Loop Optimization of An Active Prosthesis. arXiv. preprint Ch. 45 Ch. 46
- (2023). Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes. International Conference on Machine Learning. Ch. 46 Ch. 47
- (2024). A Multilab Replication of the Induced-Compliance Paradigm of Cognitive Dissonance. Advances in Methods and Practices in Psychological Science. Ch. 45
- (2020). Stability and change of basic personal values in early adolescence: A 2‐year longitudinal study. Journal of Personality. Ch. 45 Ch. 47
- (2018). Stronger shared taste for natural aesthetic domains than for artifacts of human culture. Cognition. Ch. 45 Ch. 46 Ch. 47
- (2020). On the equivalence of optimal recommendation sets and myopically optimal query sets. Artificial Intelligence. Ch. 45
- (2021). A Multisite Preregistered Paradigmatic Test of the Ego-Depletion Effect. Psychological Science. Ch. 45
- (2022). Personalizing over-the-counter hearing aids using pairwise comparisons. Smart Health. doi:10.1016/j.smhl.2021.100231. Ch. 47
- (2024). The Effects of Generative AI on Design Fixation and Divergent Thinking. Proceedings of the CHI Conference on Human Factors in Computing Systems. Ch. 46
- (2025). On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback. ICLR 2025. Ch. 45 Ch. 46
- (2024). Stopping Bayesian Optimization with Probabilistic Regret Bounds. NeurIPS 2024. Ch. 46 Ch. 47
- (2026). Knowledge Gradient for Preference Learning. arXiv. preprint Ch. 45 Ch. 47
- (2026). Cost-aware Stopping for Bayesian Optimization. International Conference on Machine Learning. Ch. 46 Ch. 47
- (2024b). Principled Preferential Bayesian Optimization. International Conference on Machine Learning. Ch. 45
- (2017). The effect of mood on judgments of subjective well-being: Nine tests of the judgment model. Journal of Personality and Social Psychology. Ch. 45
- (2025). Loss aversion is not robust: A re-meta-analysis. Journal of Economic Psychology. Ch. 45
- (2025a). PABBO: Preferential Amortized Black-Box Optimization. ICLR 2025. Ch. 45
- (2026b). Vibrotactile Preference Learning: Uncertainty-Aware Preference Learning for Personalized Vibration Feedback. UMAP 2026 (per Semantic Scholar). Ch. 45
- (2024). Value construction through sequential sampling explains serial dependencies in decision making. eLife. doi:10.7554/eLife.96997. Ch. 45 Ch. 46 Ch. 47