Suggestions for Implementing Multi-fidelity models in GPyTorch

[Abhijnan03]

I am trying to implement multi-fidelity models, specifically models like co-kriging in GPyTorch. I wanted to ask if there are any suggestions for implementing multi-fidelity models in GPyTorch. Are there any particular parts of GPyTorch which will be helpful or any tutorials that I can look at for suggestions?

[gpleiss]

What types of multi fidelity models are you interested in? I am not too familiar with the literature, but I know that people have proposed likelihoods/deep models/etc. to handle multi-fidelity modeling.

If you point us towards a particular class of models that you are interested in, we could then offer some guidance for how to implement them.

[Abhijnan03]

I am interested in implementing the multi-fidelity model proposed in the attached paper. It is represented by equation 3.1 in the paper under the heading "Co-kriging". It is essentially the summation of two different Gaussian Processes and the paper even details what the covariance matrix should look like for the model. But I wasn't sure how to best implement this in GPyTorch so I was looking for suggestions for this particular model.
forrester 2007.pdf

[eytan]

You might be interested in checking out some of the multi task models used in botorch. We typically use the ICM kernel for MF modeling (although in some cases one can just encode the fidelity as another spatial input into the kernel). https://botorch.org/docs/models

…

On Sat, Oct 22, 2022 at 8:07 PM Abhijnan03 ***@***.***> wrote: I am interested in implementing the multi-fidelity model proposed in the attached paper. It is represented by equation 3.1 in the paper under the heading "Co-kriging". It is essentially the summation of two different Gaussian Processes and the paper even details what the covariance matrix should look like for the model. But I wasn't sure how to best implement this in GPyTorch so I was looking for suggestions for this particular model. forrester 2007.pdf <https://github.com/cornellius-gp/gpytorch/files/9845483/forrester.2007.pdf> — Reply to this email directly, view it on GitHub <#2165 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAW34K36QIH6V3CZBAI6KDWER6TDANCNFSM6AAAAAARKNCFF4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>

[Abhijnan03]

Thank you for the help. Those models do indeed look useful for implementing multifidelity models. I was also wondering if there is any way to implement a multifidelity model that uses a linear autoregressive approach as opposed to a coregionalized approach?

[stdrj]

Hi Abhijnan, I wonder are there any updates on the linear autoregressive approach you mentioned in the reply. I'm also encountering a similar situation: I mainly use botorch for GP and Bayesian Optimization, but botorch doesn't support the linear autoregressive approach for multifidelity modeling.