Bugfix Release

Bug fixes

• Fixed issue with broken wheel build (#444).

Other changes

• Changed code style to use absolute imports throughout (#443).

Bugfix Release

Bug fixes

• There was a mysterious issue with the 0.2.3 wheel on pypi, where part of the botorch/optim/utils.py file was not included, which resulted in an ImportError for many central components of the code. Interestingly, the source dist (built with the same command) did not have this issue.
• Preserve order in ChainedOutcomeTransform (#440).

New Features

• Utilities for estimating the feasible volume under outcome constraints (#437).

Pairwise GP for Preference Learning, Sampling Strategies

Introduces a new Pairwise GP model for Preference Learning with pair-wise preferential feedback, as well as a Sampling Strategies abstraction for generating candidates from a discrete candidate set.

Compatibility

• Require PyTorch >=1.5 (#423).
• Require GPyTorch >=1.1.1 (#425).

New Features

• Add PairwiseGP for preference learning with pair-wise comparison data (#388).
• Add SamplingStrategy abstraction for sampling-based generation strategies, including
  MaxPosteriorSampling (i.e. Thompson Sampling) and BoltzmannSampling (#218, #407).

Deprecations

• The existing botorch.gen module is moved to botorch.generation.gen and imports
  from botorch.gen will raise a warning (an error in the next release) (#218).

Bug fixes

• Fix & update a number of tutorials (#394, #398, #393, #399, #403).
• Fix CUDA tests (#404).
• Fix sobol maxdim limitation in prune_baseline (#419).

Other changes

• Better stopping criteria for stochastic optimization (#392).
• Improve numerical stability of LinearTruncatedFidelityKernel (#409).
• Allow batched best_f in qExpectedImprovement and qProbabilityOfImprovement
  (#411).
• Introduce new logger framework (#412).
• Faster indexing in some situations (#414).
• More generic BaseTestProblem (9e604fe).

Require Python 3.7 and new features

Require Python 3.7 and adds new features for active learning and multi-fidelity optimization, along with a number of bug fixes.

Compatibility

• Require PyTorch >=1.4 (#379).
• Require Python >=3.7 (#378).

New Features

• Add qNegIntegratedPosteriorVariance for Bayesian active learning (#377).
• Add FixedNoiseMultiFidelityGP, analogous to SingleTaskMultiFidelityGP (#386).
• Support scalarize_posterior for m>1 and q>1 posteriors (#374).
• Support subset_output method on multi-fidelity models (#372).
• Add utilities for sampling from simplex and hypersphere (#369).

Bug fixes

• Fix TestLoader local test discovery (#376).
• Fix batch-list conversion of SingleTaskMultiFidelityGP (#370).
• Validate tensor args before checking input scaling for more
  informative error messaages (#368).
• Fix flaky qNoisyExpectedImprovement test (#362).
• Fix test function in closed-loop tutorial (#360).
• Fix num_output attribute in BoTorch/Ax tutorial (#355).

Other changes

• Require output dimension in MultiTaskGP (#383).
• Update code of conduct (#380).
• Remove deprecated joint_optimize and sequential_optimize (#363).

Compatibility Release

Minor bug fix release.

New Features

• Add a static method for getting batch shapes for batched MO models (#346).

Bug fixes

• Revamp qKG constructor to avoid issue with missing objective (#351).
• Make sure MVES can support sampled costs like KG (#352).

Other changes

• Allow custom module-to-array handling in fit_gpytorch_scipy (#341).

Max-value entropy search, multi-fidelity (cost-aware) optimization

This release adds the popular Max-value Entropy Search (MES) acquisition function, as well as support for multi-fidelity Bayesian optimization via both the Knowledge Gradient (KG) and MES.

Compatibility

• Require PyTorch >=1.3.1 (#313).
• Require GPyTorch >=1.0 (#342).

New Features

• Add cost-aware KnowledgeGradient (qMultiFidelityKnowledgeGradient) for multi-fidelity optimization (#292).
• Add qMaxValueEntropy and qMultiFidelityMaxValueEntropy max-value entropy search acquisition functions (#298).
• Add subset_output functionality to (most) models (#324).
• Add outcome transforms and input transforms (#321).
• Add outcome_transform kwarg to model constructors for automatic outcome transformation and un-transformation (#327).
• Add cost-aware utilities for cost-sensitive acquisiiton functions (#289).
• Add DeterminsticModel and DetermisticPosterior abstractions (#288).
• Add AffineFidelityCostModel (f838eac).
• Add project_to_target_fidelity and expand_trace_observations utilities for use in multi-fidelity optimization (1ca12ac).

Performance Improvements

• New prune_baseline option for pruning X_baseline in qNoisyExpectedImprovement (#287).
• Do not use approximate MLL computation for deterministic fitting (#314).
• Avoid re-evaluating the acquisition function in gen_candidates_torch (#319).
• Use CPU where possible in gen_batch_initial_conditions to avoid memory issues on the GPU (#323).

Bug fixes

• Properly register NoiseModelAddedLossTerm in HeteroskedasticSingleTaskGP (671c93a).
• Fix batch mode for MultiTaskGPyTorchModel (#316).
• Honor propagate_grads argument in fantasize of FixedNoiseGP (#303).
• Properly handle diag arg in LinearTruncatedFidelityKernel (#320).

Other changes

• Consolidate and simplify multi-fidelity models (#308).
• New license header style (#309).
• Validate shape of best_f in qExpectedImprovement (#299).
• Support specifying observation noise explicitly for all models (#256).
• Add num_outputs property to the Model API (#330).
• Validate output shape of models upon instantiating acquisition functions (#331).

Tests

• Silence warnings outside of explicit tests (#290).
• Enforce full sphinx docs coverage in CI (#294).

Knowledge Gradient (one-shot), various maintenance

Breaking Changes

• Require explicit output dimensions in BoTorch models (#238)
• Make joint_optimize / sequential_optimize return acquisition function
  values (#149) [note deprecation notice below]
• standardize now works on the second to last dimension (#263)
• Refactor synthetic test functions (#273)

New Features

• Add qKnowledgeGradient acquisition function (#272, #276)
• Add input scaling check to standard models (#267)
• Add cyclic_optimize, convergence criterion class (#269)
• Add settings.debug context manager (#242)

Deprecations

• Consolidate sequential_optimize and joint_optimize into optimize_acqf
  (#150)

Bug fixes

• Properly pass noise levels to GPs using a FixedNoiseGaussianLikelihood (#241)
  [requires gpytorch > 0.3.5]
• Fix q-batch dimension issue in ConstrainedExpectedImprovement
  (6c06718)
• Fix parameter constraint issues on GPU (#260)

Minor changes

• Add decorator for concatenating pending points (#240)
• Draw independent sample from prior for each hyperparameter (#244)
• Allow dim > 1111 for gen_batch_initial_conditions (#249)
• Allow optimize_acqf to use q>1 for AnalyticAcquisitionFunction (#257)
• Allow excluding parameters in fit functions (#259)
• Track the final iteration objective value in fit_gpytorch_scipy (#258)
• Error out on unexpected dims in parameter constraint generation (#270)
• Compute acquisition values in gen_ functions w/o grad (#274)

Tests

• Introduce BotorchTestCase to simplify test code (#243)
• Refactor tests to have monolithic cuda tests (#261)

Compatibility and Maintenance release

Compatibility

• Updates to support breaking changes in PyTorch to boolean masks and tensor
  comparisons (#224).
• Require PyTorch >=1.2 (#225).
• Require GPyTorch >=0.3.5 (itself a compatibility release).

New Features

• Add FixedFeatureAcquisitionFunction wrapper that simplifies optimizing
  acquisition functions over a subset of input features (#219).
• Add ScalarizedObjective for scalarizing posteriors (#210).
• Change default optimization behavior to use L-BFGS-B by for box constraints
  (#207).

Bug fixes

• Add validation to candidate generation (#213), making sure constraints are
  strictly satisfied (rater than just up to numerical accuracy of the optimizer).

Minor changes

• Introduce AcquisitionObjective base class (#220).
• Add propagate_grads context manager, replacing the propagate_grads kwarg in
  model posterior() calls (#221)
• Add batch_initial_conditions argument to joint_optimize() for
  warm-starting the optimization (ec3365a).
• Add return_best_only argument to joint_optimize() (#216). Useful for
  implementing advanced warm-starting procedures.

Maintenance release

Bug fixes

• Avoid PyTorch bug resulting in bad gradients on GPU by requiring GPyTorch >= 0.3.4
• Fixes to resampling behavior in MCSamplers (#204)

Experimental Features

• Linear truncated kernel for multi-fidelity bayesian optimization (#192)
• SingleTaskMultiFidelityGP for GP models that have fidelity parameters (#181)

API updates, more robust model fitting

Breaking changes

• rename botorch.qmc to botorch.sampling, move MC samplers from
  acquisition.sampler to botorch.sampling.samplers (#172)

New Features

• Add condition_on_observations and fantasize to the Model level API (#173)
• Support pending observations generically for all MCAcqusitionFunctions (#176)
• Add fidelity kernel for training iterations/training data points (#178)
• Support for optimization constraints across q-batches (to support things like
  sample budget constraints) (2a95a6c)
• Add ModelList <-> Batched Model converter (#187)
• New test functions
  • basic: neg_ackley, cosine8, neg_levy, neg_rosenbrock, neg_shekel
    (e26dc75)
  • for multi-fidelity BO: neg_aug_branin, neg_aug_hartmann6,
    neg_aug_rosenbrock (ec4aca7)

Improved functionality:

• More robust model fitting
  • Catch gpytorch numerical issues and return NaN to the optimizer (#184)
  • Restart optimization upon failure by sampling hyperparameters from their prior (#188)
  • Sequentially fit batched and ModelListGP models by default (#189)
  • Change minimum inferred noise level (e2c64fe)
• Introduce optional batch limit in joint_optimize to increases scalability of
  parallel optimization (baab578)
• Change constructor of ModelListGP to comply with GPyTorch’s IndependentModelList
  constructor (a6cf739)
• Use torch.random to set default seed for samplers (rather than random) to
  making sampling reproducible when setting torch.manual_seed
  (ae507ad)

Performance Improvements

• Use einsum in LinearMCObjective (22ca295)
• Change default Sobol sample size for MCAquisitionFunctions to be base-2 for
  better MC integration performance (5d8e818)
• Add ability to fit models in SumMarginalLogLikelihood sequentially (and make
  that the default setting) (#183)
• Do not construct the full covariance matrix when computing posterior of
  single-output BatchedMultiOutputGPyTorchModel (#185)

Bug fixes

• Properly handle observation_noise kwarg for BatchedMultiOutputGPyTorchModels (#182)
• Fix a issue where f_best was always max for NoisyExpectedImprovement
  (410de58)
• Fix bug and numerical issues in initialize_q_batch
  (844dcd1)
• Fix numerical issues with inv_transform for qMC sampling (#162)

Other

• Bump GPyTorch minimum requirement to 0.3.3