Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Whitening parametrisation #60

Closed
st-- opened this issue Sep 28, 2021 · 8 comments · Fixed by #71
Closed

Whitening parametrisation #60

st-- opened this issue Sep 28, 2021 · 8 comments · Fixed by #71

Comments

@st--
Copy link
Member

st-- commented Sep 28, 2021

In many cases, optimising an SVGP model is much easier when parametrising it in the "whitened" representation, i.e. for L L' = K, we parametrise u = f(Z) = L * v with v ~ N(0, I) (such that under the prior, u ~ N(0, K) is the same as before). Could maybe make use of PDMats.jl's whiten/unwhiten functions. Not sure how this would work as a feature within the package, but at the very least it should be fairly straightforward to demonstrate how you would do this "by hand" in one of the notebooks.

@willtebbutt
Copy link
Member

Agreed -- this parametrisation would be very helpful. Starting with a notebook and then figuring out how it might work inside the package seems good to me. Probably we just need a way inside the SVGP to specify which parametrisation we're using so that we can dispatch on that or something.

@rossviljoen
Copy link
Collaborator

I've also been messing around with MCMC for sparse GPs which uses the same parameterisation - so would be good to figure out a general way of handling it

@theogf
Copy link
Member

theogf commented Nov 5, 2021

@rossviljoen Do you have an example of this somewhere? Even as a gist?

@rossviljoen
Copy link
Collaborator

yes, it's an example in the branch ross/mcmc of this repo (not sure it actually belongs in this package though?)

@willtebbutt
Copy link
Member

Am working on doing this at the minute. PR incoming shortly.

@theogf
Copy link
Member

theogf commented Nov 5, 2021

Hahaha so we are only 3 people working on exactly the same thing 😆

@theogf
Copy link
Member

theogf commented Nov 5, 2021

Mine is working already 😈

@rossviljoen
Copy link
Collaborator

We'll have to do double blind PR review 😆

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants