Computing GP posterior predictive given noisy & noiseless observations

[mjoerke]

I'm interested in implementing an acquisition function from Bayesian Algorithm Execution (BAX) in GPyTorch. BAX has existing code in gpflow, but I have a large amount of existing code already written in GPyTorch.

The acquisition function I'm interested in involves computing the posterior predictive conditioned both on noisy observations and noiseless observations (see Appendix A.1, pg 24). Is it possible to implement something like this in GPytorch?

Thank you!

[Balandat]

In general it should be, I'd have to look a bit more at the details here. @KeAWang has contributed to gpytorch and should be vaguely familiar with the paper :) - maybe he can provide some input here

[KeAWang]

If you want to condition a GP on both noisy and noiseless observations, you just need to keep track of which indices are noiseless. See Ch 2 of the GPML book. Instead of adding a noise term to every entry of the diagonal, you only add to the subset of diagonal entries that correspond to indices with observation noise.

You can get what you want by modifying FixedNoiseGaussianLikelihood

gpytorch/gpytorch/likelihoods/gaussian_likelihood.py

Line 167 in 1c490dd

class FixedNoiseGaussianLikelihood(_GaussianLikelihoodBase):

with learn_additional_noise=True. All you have to do is mask out the noiseless entries when adding second_noise.