Software, Evaluation, and the Research Community

The previous chapters reported what methods for preferential Bayesian optimization (PBO) claim. This chapter turns to the conditions under which those claims were made and can be used: the software that implements the methods, the defaults it chooses on the user's behalf, the way papers evaluate one method against another, and the people who do the work.

The picture as of September 2026 has three parts. There is one maintained, general-purpose software stack for PBO, Meta's BoTorch and Ax; the rest is mostly code released with single papers. Evaluation relies almost entirely on simulated users answering comparisons about synthetic test functions inherited from scalar optimization. And the research community is small and spread over machine learning, control, robotics, and human-computer interaction, which publish in different venues and, in the cross-citations we checked, rarely cite one another.

31.1 Software #

A reader who wants to run PBO today has to choose software, and the choice is narrower than the size of the Bayesian optimization ecosystem suggests. Dates below are those in the projects' changelogs (the files in which a project lists what changed in each version) or, where a project has no changelog entry, upload dates on the Python Package Index (PyPI); the two can differ by a day.

31.1.1 BoTorch #

BoTorch is Meta's library of Gaussian process models and acquisition functions built on PyTorch; Section 18.6 and Section 27.5 described its preference model, PairwiseGP. Its preference features arrived between 2020 and 2024 (Meta Platforms, Inc., 2026e):

• 0.2.3 (April 27, 2020) added PairwiseGP for pairwise comparison data.
• 0.3.2 (October 2020) removed its noise term and added a ScaleKernel by default.
• 0.6.3 (March 28, 2022) added LearnedObjective, the analytic form of EUBO (the expected utility of the best option, Section 19.4), and a tutorial on Bayesian optimization with preference exploration (BOPE), in which a person states preferences over the outcomes of experiments rather than over designs.
• 0.6.5 (July 2022) added a logistic link likelihood and switched the PBO tutorial to EUBO.
• 0.9.0 (August 1, 2023) added pairwise BALD (Bayesian active learning by disagreement, which picks the query whose answer the model's plausible hypotheses disagree about most).
• 0.10.0 (February 26, 2024) added qEUBO, the form of EUBO for queries of $q$ options.

Since then the preference code has received only fixes: in 0.18.0 (June 3, 2026), a fix to how PairwiseGP manages its stored state and a numerically stabler mixture entropy in BALD. Version 0.18.1 (June 8, 2026) is the latest release, and the last commit to the main branch was on September 30, 2026 (Meta Platforms, Inc., 2026i). The preference acquisition functions live in one source file (Meta Platforms, Inc., 2026j).

BoTorch's PBO tutorial is the first code most newcomers run, so its design is worth knowing. It uses a 4-dimensional linear utility, adds Gaussian noise with standard deviation 0.1 to the utilities before comparing, evaluates the model with Kendall's rank correlation (the fraction of pairs that two rankings order the same way, rescaled to run from $-1$ to $1$), and compares EUBO with random queries over only 3 repetitions (Meta Platforms, Inc., 2026c).

31.1.2 Ax #

Ax is Meta's platform for managing experiments, which calls BoTorch for its models. A pairwise model bridge, PairwiseModelBridge, first appears in the installed package of Ax 0.2.6 (August 17, 2022) (Facebook, Inc., 2022). Ax 1.0.0 was uploaded to PyPI on May 8, 2025, with a companion paper by Olson et al. (2025) (AutoML 2025). Ax's preference features arrived in 2026 (Meta Platforms, Inc., 2026l):

• 1.2.2 (January 2026; PyPI has no files for this version) added a PreferenceOptimizationConfig with storage support and a Kendall rank correlation diagnostic.
• 1.2.3 (February 19, 2026) added BOPE utility traces based on PairwiseGP, and LLMProvider and LLMMessage abstractions for language models.
• 1.3.0 (June 4, 2026) added LILO labeling trials with a check that the labels are fresh, automatic dispatch of qEUBO for PBO, PairwiseGP inside ModelList, and a utility ranking plot. LILO turns free-text feedback into pairwise labels with a language model (Section 35.2).
• 1.3.1 (June 9, 2026) is the latest release and requires Python 3.11 or later (Meta Platforms, Inc., 2026a).

Ax's changelog begins only with 1.1.0 (August 2025), so the earlier history comes from the installed packages. The main branch (last commit September 29, 2026) has no tutorial on preferences, BOPE, or LILO (Meta Platforms, Inc., 2026m), and the LILO logic exists only in code (Meta Platforms, Inc., 2026b).

31.1.3 Every package with preference support #

Table 31.1 lists the software with preference support that we found; the links behind each name are code repositories or software documentation. HB-EI and HB-UCB are the hallucination believer of Section 19.3 with expected improvement or an upper confidence bound.

Several more methods exist only as code for a single paper: PPBO (Aalto PML, 2022) (MIT, last commit September 13, 2022); POLAR (Tucker, 2024) (MATLAB, BSD-3-Clause, last commit June 17, 2024, supporting pairwise, coactive, and ordinal feedback); the LILO research code (Meta Research, 2026) (MIT, last commit May 12, 2026); and DT-PBO, PBO with decision-tree surrogates (Quadt, 2026) (MIT). Three repositories have no license file, so default copyright applies and reuse is not permitted without asking: POP-BO (PREDICT-EPFL, 2024), MR-LPF (Kayal, 2025), and CrashPBO (Institute for Data Science in Mechanical Engineering, RWTH Aachen University, 2026).

31.1.4 Libraries without preference support #

None of the major Bayesian optimization and Gaussian process libraries has a preference likelihood or a preference acquisition function. We searched the source trees of the latest versions at the end of September 2026, GPyTorch 1.15.2 (GPyTorch developers, 2026), GPflow 2.11.1 (GPflow developers, 2026), Trieste 4.6.0 (Secondmind Labs, 2026), SMAC 2.4.1 (AutoML.org, 2026), HEBO 0.3.6 (Huawei Noah's Ark Lab, 2024), Dragonfly 0.1.7 (Dragonfly developers, 2022), GPyOpt 1.2.6 (SheffieldML, 2023), and Optuna 5.0.0 (Optuna developers, 2026a), for pairwise, preference, and dueling. PairwiseGP uses GPyTorch's kernels but implements its own Laplace inference; "dueling" appears in HEBO only in an unrelated reinforcement learning subproject; and Optuna's preference support lives in optuna-dashboard and OptunaHub. GPyOpt was archived on January 17, 2023, and Dragonfly is effectively unmaintained.

31.1.5 What the software landscape means #

Three judgments follow (inference). Practical PBO depends on one organization: BoTorch supplies the models and acquisition functions, Ax manages experiments, BOPE, and LILO, and AEPsych serves psychophysics; since qEUBO in February 2024, new preference features have landed in Ax rather than BoTorch. Alternatives to the Gaussian process (GLISp, DT-PBO, PABBO) are niche, and only GLIS can be installed with pip. And for a practitioner, Ax 1.3.x is the most complete maintained software but lacks documentation for these features; we did not verify whether Ax lets a user pass a dimension-scaled prior to PairwiseGP through its model configuration.

31.2 Default priors that ignore dimension #

A default prior is a modeling decision that most users never see. Section 27.5 read the choices inside PairwiseGP (Table 18.2 lists them), and Section 30.1 explained why a lengthscale prior of fixed scale fails as the number of inputs grows. Table 31.2 collects the defaults of every preference package we examined, with BoTorch's scalar models as the point of comparison. No row scales with dimension except the comparison row.

No PBO package documents how its default prior relates to dimension, so anyone using these packages above about 10 dimensions inherits the fixed-scale priors that scalar Bayesian optimization abandoned in 2024 (inference; Figure 30.1 shows the consequence, and Section 30.3.5 what to pass instead). This is a different question from whether a prior should be informed by other users' data, a population prior, which Section 32.5 takes up: the defaults in Table 31.2 are weak priors, but weak at the wrong scale once the dimension grows (inference).

31.3 Reproducing research code #

A method that cannot be rerun cannot be checked, and a baseline that cannot be rerun gets reimplemented, each time a little differently. Three habits make research code rerunnable: a release tag (a named, frozen version of the repository), a dependency file that lists the exact library versions the code needs, and a license that says who may reuse it.

At the end of September 2026 we checked 11 research repositories, prefGP, SkewGP, preferentialBO, the official qEUBO repository, PABBO, PPBO, POLAR, GLIS, POP-BO, LILO, and BO_toolbox, for release tags: all 11 had none. Dependencies are commonly pinned to versions from 2020 to 2024 (Table 31.1 lists several), PPBO pins numpy 1.18.4 and the archived GPyOpt 1.2.6 (Aalto PML, 2022), and neither qEUBO repository has a dependency file (Meta Research, 2023; Astudillo, 2023a). preferentialBO's scripts call np.int, which NumPy 1.24 removed, so they run only with the pinned numpy 1.23.2; its README also warns that on Ubuntu 20.04 it has not been confirmed that the paper's results can be fully reproduced (the experiments were run on CentOS 6.9) (CyberAgent AI Lab, 2023).

Licenses are not uniform either. Most repositories use MIT, BSD, or Apache licenses, but PABBO uses the AGPL-3.0, a copyleft license that requires derived software, including software offered over a network, to be released under the same license; AEPsych's Creative Commons Attribution-NonCommercial 4.0 license restricts commercial reuse (Meta, 2026); and three repositories from 2024 to 2026 have no license file at all. Each method also brings its own Gaussian process implementation (GPy, BoTorch 0.9 or 0.10, JAX, MATLAB), so most papers reimplement their baselines, which may be one reason the same baseline looks strong in one paper and weak in another (inference).

31.4 How methods are evaluated #

Comparing optimizers is harder than it looks. A single run depends on the random initial design and on the noise in the answers, so a method can win one run by luck, which is why comparisons average over many runs (Chapter 11). And a regret bound (Chapter 13) holds for a class of functions, not for the function in front of you, so only benchmarks can say how methods rank in practice, within the limits this section describes.

31.4.1 Inherited test functions and simulated users #

Since González et al. (2017), almost every PBO paper has been evaluated with a simulated user: a program that answers each comparison by evaluating a known function at the two options and adding noise. The functions are standard synthetic test functions, usually in 1 to 8 dimensions, which González et al. took from the library of scalar test functions maintained by Surjanovic and Bingham and later papers kept. These functions were designed to test scalar optimizers; they have none of the properties of human utility that Part IV and Part IX describe, such as indifference thresholds, drift, intransitivity, and anchoring (inference). Table 31.3 lists the test problems and metrics of representative papers; Table 28.3 gives the noise, budget, and conclusion of most of them, and the lists below collect the noise models and metrics.

In the table, MaxMinLCB is the max-min lower confidence bound algorithm of Pásztor et al., PF-TS is Thompson sampling with preference feedback, and HPO-B is a benchmark built from hyperparameter tuning tasks of the kind Chapter 22 works through. Papers from 2026 continue the same practice, among them KappaSharp, a preprint, with 11 benchmarks in 5 to 20 dimensions (Shao et al., 2026).

31.4.2 Noise models chosen paper by paper #

Each paper picks its own model of how the simulated user errs. We found at least six kinds:

1. Gaussian noise added to the utilities before comparing, with standard deviation 0.01 (Takeno et al.), 0.05 (Siivola et al.), 0.1 (the BoTorch tutorial), or 10% of the function's range (local PBO (Menn et al., 2026a)).
2. Logistic noise calibrated so that random pairs among the top 1% of points are answered wrongly 10%, 20%, or 30% of the time (qEUBO, in an appendix study); for 6-dimensional Ackley, these correspond to logistic scales of 0.0575, 0.1416, and 0.2943 (Astudillo, 2023b).
3. A fixed probability of flipping the answer (10% in BOPE).
4. No noise by default (PABBO).
5. Probit noise on a standardized function (Fauvel and Chalk).
6. A decision maker simulated by a language model (LILO (Kobalczyk et al., 2026)).

Calibrating to an error rate is closer to reality than setting a scale directly, because it fixes how often the simulated user is wrong on the comparisons that matter, but the noise is still homoscedastic (the same everywhere) and independent from one answer to the next (inference). Even two models matched to the same error rate on one gap disagree on every other gap, as Figure 31.1 shows.

Some things to try:

• With the defaults, read the three curves at a gap of 0.30. The flip model still errs 10% of the time, logistic noise 0.14%, and probit noise 0.006%: "10% noise" makes a clearly worse option about 70 times more likely to win under the flip model than under logistic noise, and over a thousand times more likely than under probit noise (Exercise 31.3 works the probit case by hand).
• Move the reading gap to 0.05, a closer call than the matched one. Now logistic and probit noise err 25% and 26% of the time, the flip model still 10%: the flip model is harsher on clear choices and gentler on close ones.
• Raise the error rate at the matched gap to 30%. At a gap of 0.30 the logistic and probit users now err 7.3% and 5.8% of the time, close to each other, while the flip model errs 30%. With very noisy answers the two utility-noise models nearly agree; it is the flip model that stands apart.

31.4.3 Regret, defined five ways #

Regret is the shortfall of a method's result from the best achievable value (Section 13.1). In PBO the method never sees utility values, so a paper must decide which point counts as "the result", and the papers decide differently:

• the function value at the current Condorcet winner, the point that wins most often (González et al.);
• the simple regret at the maximizer of the posterior mean, the point the model would recommend (qEUBO, BOPE; Takeno et al. use their recommended point);
• the simple regret of the best point queried so far (PABBO);
• the cumulative regret summed over both points of every comparison, in preference probabilities (MaxMinLCB, MR-LPF, PF-TS);
• the cumulative regret in utility (POP-BO).

Papers also report the quality of the model's ranking (Kendall's rank correlation in the BoTorch tutorial and in Ax), the computing time per iteration (qEUBO, PABBO, Takeno et al.), and, in a user experiment, the number of simulation steps needed to reach the nearest local minimum (Mikkola et al.). No paper uses the number of comparisons needed to reach a threshold as its main metric, although that is the quantity a person in the loop pays for.

The choice of metric changes the ranking of methods. In the POP-BO paper, qEUBO reports a slightly better solution than POP-BO on Gaussian process sample instances, but its cumulative regret is more than 2.5 times higher, and on 6-dimensional Ackley the two metrics reverse again (Xu et al., 2024b). A method that queries near the optimum is favored by regret at the queried points; a method whose model is well calibrated is favored by regret at the posterior mean (inference). Figure 31.2 lets the reader watch this happen.

Some things to try:

• With the defaults (30 runs, noise 0.1), Recommended puts EUBO first, the incumbent rule close behind with overlapping bands, and random pairs last.
• Switch to Best queried. Random pairs now win, with regret near 0.05, through coverage rather than learning: 16 random pairs make 32 draws from only 41 candidates, so on average about 23 of the candidates have been shown (Exercise 31.1). In a larger space, or in more dimensions, the same metric would not favor random pairs so strongly (inference).
• Switch to Cumulative. The incumbent rule wins clearly: every pair it asks contains the model's current best guess, which is soon near the optimum, so half of each comparison costs almost nothing (Exercise 31.2).
• Set Runs to 5 and press Another batch of runs several times. Under Recommended, the winner changes from batch to batch, mostly between EUBO and the incumbent rule: with few repetitions, the ranking is partly a draw.
• Change the noise. At 0.2, random pairs overtake the incumbent rule under Recommended, while the other two metrics keep their order.

31.4.4 Statistics and the size of the differences #

Statistical reporting is thin. Papers use from 3 repetitions (the BoTorch tutorial), 10 (Takeno et al.), 20, and 30, up to 50 to 100 (qEUBO), and show uncertainty as ±1, ±1.96, or ±2 standard errors, 95% confidence intervals, or interquartile ranges. Only Fauvel and Chalk test many hypotheses formally: across 34 functions they compare every pair of methods with a Mann-Whitney U test (a test of whether one method's results tend to be larger than another's, without assuming a distribution) and aggregate the wins into Borda scores (each method scores one point for every method it beats) (Fauvel and Chalk, 2021). No later paper adopted the approach. Where differences are measured at all, the gap between elaborate acquisition functions and random queries is often small, and Section 28.9 reports the evidence and why the existing comparisons do not add up. The practical reading: repeat runs at least tens of times, include random queries, and test the differences that a claim rests on (inference).

31.5 Real-data tasks and missing datasets #

The tasks that papers call "real data" all turn the judgments of a group into a single deterministic utility. Table 31.4 lists the four in use.

These tasks reward the method that finds the optimum of a population, and they hide the inconsistency of individual judgments, which is the very thing PBO exists to handle (inference). We found no shared benchmark suite or leaderboard for PBO, and no public data set of individual-level pairwise judgments designed for evaluating PBO. Such a data set, with timestamps, presentation order, response times, and repeated pairs, is what would let methods be compared offline on real answers (Section 46.7).

31.6 Reproduction problems found #

Checking papers against their code turned up the following problems.

• qEUBO's Animation task. The paper says that the fitted model is a support vector machine (Astudillo et al., 2023). The author's personal repository has two versions of the ground truth: animation_runner.py uses a support vector machine classifier and animation2_runner.py loads a fitted PairwiseGP (Astudillo, 2023a). The official repository the paper cites has no run script or data for the Animation task (Meta Research, 2023), and neither repository lists its dependencies.
• BOPE's dimensions. The text gives "DTLZ2 (d = 4, k = 8)" while a figure is labeled "DTLZ2 (d=8, k=4)" (Lin et al., 2022).
• Candy's size. Two papers report 86 and 85 candies (Siivola et al., 2021; Zhang et al., 2025a); FiveThirtyEight's data file has 85 rows (FiveThirtyEight, 2017).
• Takeno et al.'s code does not run as is in a current environment (Section 31.3) (CyberAgent AI Lab, 2023).

None of these is large on its own; together they mean that the benchmark results of PBO cannot at present be reproduced one by one without contacting the authors (inference).

31.7 Simulated and real users #

Every benchmark above assumes that a simulated user stands in for a person well enough to rank methods. The few studies that compared the two directly all found clear differences. Table 31.5 collects them.

The table's numbers carry the main point, and three details add to it. In the retinal implant study, which tested an optimization that Granley et al. (2023) (NeurIPS 2023) had evaluated only in simulation, 16 of the 17 participants still preferred the optimized result under the main condition, and the authors stress "the importance of validating optimization strategies with human participants" (Schoinas et al., 2025). In Ou et al.'s three-month field deployment, 415 of 549 evaluation sequences stopped at the first iteration, and the authors conclude that "optimization using preferential choices lacks mechanisms to deal with inconsistent and contradictory human judgments", and that "machine outcomes, in turn, influence future user inputs via heuristic biases and loss aversion" (Ou et al., 2022) (Section 32.3, Section 32.6). And users who understand how the optimizer works "strategically provide biased answers" (Colella et al., 2020), contrary to the simulated user's assumption that feedback is faithful, while Mikkola et al. report that their method could tell choices made by people from those made by a computer program (Mikkola et al., 2020).

None of these behaviors appears in any simulated-user benchmark we found (inference). Simulated evaluation can compare how algorithms behave under a given noise model, but it cannot predict how they rank with people (Section 28.9 describes the missing experiment). Until simulators are validated on individual data, a result obtained with a simulated user should be checked on at least a few real people before it is reported as a result about people (inference); the norms that experimental economics adopted for such comparisons are a useful model (Section 40.12).

31.8 The research community #

PBO is the work of a small number of groups. This section maps them by discipline, then reports the infrastructure of a field (surveys, theses, workshops, venues) and how much is published.

31.8.1 Groups by discipline #

Table 31.6 arranges the groups by the discipline they publish in, which is also, largely, the kind of question they ask.

Other recurring contributors include Mesbah and colleagues (KappaSharp, a 2026 preprint) (Shao et al., 2026), Wu and Gardner (a knowledge gradient for preference learning, a 2026 preprint) (Wu and Gardner, 2026), and Theiner, Hirt, Findeisen, and colleagues in control (ECC 2025 and ECC 2026) (Theiner et al., 2025; Theiner et al., 2026); we did not verify their institutions. Read across the table, the groups are moving from methods toward applications and toward language models as intermediaries, and the newcomers (Trimpe, Mesbah) come from control (inference). The groups that prove regret bounds do not run human studies, the groups that run human studies rarely change the acquisition function, and the groups that build the software sit closest to the decision-theoretic line (inference).

31.8.2 No survey, nine theses, no dedicated workshop #

There is no dedicated survey of PBO. General references on Bayesian optimization give preferences a page at most: Frazier's tutorial (Frazier, 2018) does not contain the words "preference", "pairwise", or "duel", Garnett's monograph (Garnett, 2023) mentions optimizing human preferences in about one paragraph, and the survey of Wang et al. (2023b) (ACM Computing Surveys 2023) has no section on PBO. The dueling bandit survey of Bengs et al. (2021) (JMLR 2021) covers preference probabilities modeled with Gaussian processes but cites neither González et al. 2017 nor Brochu et al. (Brochu et al., 2010). The closest reference work is the tutorial of Benavoli and Azzimonti (2026a) (Foundations and Trends in Machine Learning 2026), which covers nine Gaussian process models of preferences and choices but mentions PBO only briefly. The remaining teaching material consists of software tutorials and the book chapter Computational Design with Crowds by Koyama and Igarashi (Koyama and Igarashi, 2018), a preprint whose formal publication details we did not verify.

Doctoral theses carry much of the field's synthesis. We found at least nine doctoral theses from 2017 to 2025 that center on PBO or devote a substantial part to it (Table 31.7).

The closest workshops covered preference learning as a whole. The ICML 2023 workshop The Many Facets of Preference-Based Learning covered dueling bandits, reinforcement learning from human feedback (RLHF, the method used to align language models with human preferences), social choice, and optimization, with two papers on Bayesian optimization from preferences (ICML, 2023). PBO work has also appeared at the ICML 2019 Workshop on Human in the Loop Learning (McCourt and Dewancker, 2019), at the NeurIPS 2022 Workshop on Gaussian Processes, Spatiotemporal Modeling, and Decision-making Systems (Takeno et al., 2022), and at ProbML 2026, a symposium held alongside ICML with archival proceedings (ProbML, 2026). No workshop dedicated to PBO appeared at NeurIPS or ICML from 2024 to 2026, as far as arXiv comments and three web searches show.

31.8.3 How much is published #

Publication volume is growing, from a small base. We ran four searches, two over arXiv abstracts (arXiv, 2026b; arXiv, 2026a) and two over Semantic Scholar (Semantic Scholar, 2026), each in a narrow form (the phrase) and a broad form (preference words together with Bayesian optimization). Figure 31.3 shows the yearly counts.

All four searches show growth by a factor of about 2.4 to 4.3 from 2023 to 2025, and the first nine months of 2026 nearly match all of 2025 (12 entries against 13, 38 against 39, 10 against 13, and 32 against 34). But even in 2025 and 2026 a year brings only 10 to 40 entries. The growth coincides with the attention that language-model alignment has brought to preference feedback (inference). The cross-citations we checked are few: the 2021 dueling bandit survey does not cite González et al. 2017, and the 2026 local PBO paper does not cite the work on high-dimensional lengthscale priors (Section 30.4) (inference).

31.9 Settled, contested, missing #

31.10 Exercises #

Further reading #

• The source of PairwiseGP (Meta Platforms, Inc., 2026h) and BoTorch's PBO tutorial (Meta Platforms, Inc., 2026c) are the de facto definition of PBO in practice; read them before trusting a default.
• Fauvel and Chalk (2021) is the one paper that evaluates many acquisition functions with formal multiple comparisons; its protocol is worth copying.
• Xu et al. (2024b) shows within one paper how the metric decides which method looks better.
• Schoinas et al. (2025) and Ou et al. (2022) are the clearest direct comparisons of simulated users and real people in preferential loops.
• Benavoli and Azzimonti (2026a) is the closest thing to a reference work on the models, though not on the optimization.