Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for local variables in RenyiELBO #1608

Merged
merged 19 commits into from
Jul 10, 2023

Conversation

fehiepsi
Copy link
Member

@fehiepsi fehiepsi commented Jun 16, 2023

Fixes #1607

Also fixes a jaxns issue causing master branch CI failing.

TODO:

  • make sure that subsample_idx is the same across local variables

@fehiepsi fehiepsi requested a review from martinjankowiak June 16, 2023 19:24
@fehiepsi fehiepsi removed the request for review from martinjankowiak June 16, 2023 20:48
@fehiepsi fehiepsi added the WIP label Jun 16, 2023
numpyro/contrib/nested_sampling.py Outdated Show resolved Hide resolved
numpyro/infer/elbo.py Outdated Show resolved Hide resolved
numpyro/infer/elbo.py Outdated Show resolved Hide resolved
numpyro/infer/elbo.py Outdated Show resolved Hide resolved
numpyro/infer/elbo.py Outdated Show resolved Hide resolved
numpyro/infer/elbo.py Show resolved Hide resolved
test/infer/test_svi.py Outdated Show resolved Hide resolved
@fehiepsi
Copy link
Member Author

Thanks for reviewing, @martinjankowiak! Let me know if you have further questions.

numpyro/infer/elbo.py Outdated Show resolved Hide resolved
numpyro/infer/elbo.py Outdated Show resolved Hide resolved
martinjankowiak
martinjankowiak previously approved these changes Jul 1, 2023
Copy link
Collaborator

@martinjankowiak martinjankowiak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm but it's hard to reason about how these changes might affect the dice bits (i.e. for non-reparameterizable guide sites). i think it'd be good to add a simple test, e.g. check that if you optimize renyi elbo the result is approximately 0.0 for model = guide with model = Categorical(...). or do we have other tests that cover the non-reparameterizable case?

edit: maybe test_svi.py::test_svi_discrete_latent is sufficient? wdyt?

@fehiepsi
Copy link
Member Author

Currently, we don't support discrete latent sites in those ELBO, RenyiELBO,... A warning will be displayed in those cases (covered in test_svi_discrete_latent)

@fehiepsi fehiepsi force-pushed the renyi-local branch 2 times, most recently from e988bb1 to 59081d7 Compare July 10, 2023 14:02
@fehiepsi fehiepsi requested a review from martinjankowiak July 10, 2023 15:23
numpyro/infer/elbo.py Outdated Show resolved Hide resolved
@martinjankowiak martinjankowiak merged commit 4799b84 into pyro-ppl:master Jul 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RenyiELBO should respect local latent variables
2 participants