Confusion & errors about inducing points in variational inference example

[stevenasheets]

I've been looking at the 'Approximate_GP_Objective_Functions' notebook example and trying to extend it towards multiple input dimensions. However, I keep on running into a Runtime Error that I do not understand.

x = torch.randn((10,2))`
y = torch.randn((10,1))

And define a class with ARD:

class ApproximateGPModel(gpytorch.models.ApproximateGP):
    def __init__(self, inducing_points,ard_dim):
        variational_distribution = gpytorch.variational.CholeskyVariationalDistribution(inducing_points.size(-1))
        variational_strategy = gpytorch.variational.VariationalStrategy(
            self, inducing_points, variational_distribution, learn_inducing_locations=True
        )
        super().__init__(variational_strategy)
        self.mean_module = gpytorch.means.ConstantMean()
        self.covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.RBFKernel(ard_num_dims=ard_dim))

    def forward(self, x):
        mean_x = self.mean_module(x)
        covar_x = self.covar_module(x)
        return gpytorch.distributions.MultivariateNormal(mean_x, covar_x)

and then training the model:

inducing_pts = torch.randn((5,2))
print(inducing_pts.shape)
model = ApproximateGPModel(inducing_pts, x.shape[-1])
likelihood = gpytorch.likelihoods.GaussianLikelihood()
objective_function = gpytorch.mlls.PredictiveLogLikelihood(likelihood, model, num_data=y.numel())
optimizer = torch.optim.Adam(list(model.parameters()) + list(likelihood.parameters()), lr=0.1)

model.train()
likelihood.train()
for _ in range(training_iterations):
    output = model(x)
    loss = -objective_function(output, y)
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

`
model.eval()
likelihood.eval()
with torch.no_grad():
    f_dist = model(x)
    mean = f_dist.mean
    f_lower, f_upper = f_dist.confidence_region()
    y_dist = likelihood(f_dist)
    y_lower, y_upper = y_dist.confidence_region()`

I hit a runtime error

RuntimeError: mat1 and mat2 shapes cannot be multiplied (10x5 and 2x1)

when f_dist = model(x) is evaluated. Any ideas what I'm doing wrong?

Using gpytorch v1.9.1

[gpleiss]

The shape of y is wrong. It should be of size 10, not 10 x 1.