-
Notifications
You must be signed in to change notification settings - Fork 107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding some basic VI approximation and fitting routine #397
Comments
In #392 we define a |
@rlouf We assume the |
Great! No there is no such assumption in the library (or at least shouldn't be), we try to support PyTree states as much as we can. |
To follow the design principle of blackjax, I believe VI should also has an API of the form below? : new_state, info = kernel(rng_key, state) which would perform one optimization step for the ELBO. |
As you can see with the pathfinder implementation, Blackjax treats VI differrently from MCMC algorothms. The idea is that you first fit an approximation to the target density, and then sample from this approximation with something like (in peudo-code): approx, info = approximate(rng_key, position)
samples = sample(sample_key, approx, num_samples) I think at the higher-level the API will always be something more or less like this. We can consider a kernel-like lower interface for some algorithms if it makes sense. But again, I am no VI expert and open to suggestions. |
Can someone then, give a minimal working example for the Mean Field VI? This would be helpful also for the refactoring of the pathfinder API in #465 and the implementation of the full rank approach, i believe. |
I understand. Altough i would argue that it would be helpful to get a foot in the door, if one wants to help to develop VI further. Could be as easy as having a multivariate normal and evaluating mean field and full rank. For the current implementation i don't see immediately how the pseudo-code you provided is implemented in the library. |
@LarsKarbach I understand your point and I really wish there could be a |
Copying over from #392 (comment)
After #392, we should add the 2 most basic VI algorithm: meanfield and full rank ADVI [1]. Below is a working example of Meanfield ADVI:
Fitting a model looks like:
[1] https://arxiv.org/abs/1603.00788
The text was updated successfully, but these errors were encountered: