Bayesian optimization with derivatives in botorch

[samuela]

I'm interested in doing Bayesian optimization with derivatives in botorch. Something along the lines of https://arxiv.org/abs/1703.04389. I could've sworn I saw a tutorial page somewhere with the some example code, but hours of googling has left me empty-handed...

Does anyone know if this is possible in botorch? What would be required to implement it?

EDIT: found it here: https://github.com/cornellius-gp/gpytorch/blob/master/examples/08_Advanced_Usage/Simple_GP_Regression_Derivative_Information_1d.ipynb Is it possible to interface with this from botorch? What exactly is the line between botorch and gpytorch anyways?

[Balandat]

Yes, this is possible. There was a discussion on this as part of #636. You may be interested in particular in the demo notebook linked in my response here. It probably wouldn't be a terrible idea to make a tutorial out of this...

What exactly is the line between botorch and gpytorch anyways?

BoTorch is built as a Bayesian Optimization library that focuses primarily on making it easy to build MC acquisition functions and optimize them using PyTorch auto-differentiation. Its APIs are relatively generic and designed to work with any probabilistic PyTorch model that allows to produce posterior samples through which one can backprop (e.g. you could also hook up some kind of BNN instead of a GP as your model). GPyTorch focuses on implementing all kinds of scalable GP models. BoTorch comes pre-packaged with a bunch of GPyTorch models to make it easier and less cumbersome for users to set up their problem.

[samuela]

Awesome, thanks so much for the pointers! This is super helpful