Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up Rt infections renewal model #204

Merged
merged 3 commits into from
Jun 20, 2024
Merged

Conversation

damonbayer
Copy link
Collaborator

Cleaning up this model in preparation for #202.

Copy link

codecov bot commented Jun 18, 2024

Codecov Report

Attention: Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.

Project coverage is 92.51%. Comparing base (2eeddc6) to head (8d82b48).
Report is 2 commits behind head on main.

Files Patch % Lines
model/src/pyrenew/latent/infectionswithfeedback.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #204      +/-   ##
==========================================
- Coverage   94.76%   92.51%   -2.26%     
==========================================
  Files          40       40              
  Lines         898      895       -3     
==========================================
- Hits          851      828      -23     
- Misses         47       67      +20     
Flag Coverage Δ
unittests 92.51% <92.85%> (-2.26%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@gvegayon gvegayon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Main comment: Why is the test image being removed?

model/Makefile Show resolved Hide resolved
model/src/pyrenew/latent/infections.py Show resolved Hide resolved
@damonbayer
Copy link
Collaborator Author

damonbayer commented Jun 19, 2024

Main comment: Why is the test image being removed?

  1. The test was failing and can't get the image I produce locally to match the one produced in CI.
  2. I don't really understand the point of the test.
  3. If we want to keep a version of the test, we should just evaluate the underlying data the figure is based on, not the actual image file.

@AFg6K7h4fhy2
Copy link
Collaborator

Main comment: Why is the test image being removed?

1. The test was failing and can't get the image I produce locally to match the one produced in CI.

2. I don't really understand the point of the test.

3. If we want to keep a version of the test, we should just evaluate the underlying data the figure is based on, not the actual image file.

Agree on point 3 unless there is something I am missing @gvegayon .

@gvegayon
Copy link
Member

Main comment: Why is the test image being removed?

1. The test was failing and can't get the image I produce locally to match the one produced in CI.

2. I don't really understand the point of the test.

3. If we want to keep a version of the test, we should just evaluate the underlying data the figure is based on, not the actual image file.

Agree on point 3 unless there is something I am missing @gvegayon .

I 100% Agree with both, so then, instead of removing the test completely, I would drop in a replacement test that checks the data under the figure.

@damonbayer
Copy link
Collaborator Author

I'm just not yet sure what a good test would be. We don't test anything related to the posterior anywhere else, so I would be more inclined to push "develop tests related to posterior inference" to a separate issue.

@gvegayon
Copy link
Member

How about saving the data that it generates now? Just like the image approach, but with model outcomes

@damonbayer
Copy link
Collaborator Author

damonbayer commented Jun 20, 2024

Willing to try it, but I worry that we could get test failures based on very minor package updates. I'm also not sure if the rng would lead to the same output on all platforms.

There is a Teams thread about this somewhere (for Stan). I'll try to find it.

Edit: https://teams.microsoft.com/l/message/19:[email protected]/1717450231741?tenantId=9ce70869-60db-44fd-abe8-d2767077fc8f&groupId=4f34eba7-8fcb-4a26-9b34-a434ea777f0c&parentMessageId=1717450231741&teamName=CFA-Predict&channelName=Help&createdTime=1717450231741

Copy link
Collaborator

@AFg6K7h4fhy2 AFg6K7h4fhy2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finished reviewing the remaining files. Other than what is mentioned in #206 and maybe just a reminder for us to address the plot testing or to make a final decision on testing the data directly instead of the plotted data, this PR looks useful and appropriate.

@damonbayer
Copy link
Collaborator Author

damonbayer commented Jun 20, 2024

Based on the Teams discussion I referenced, I believe this could be merged (but I would need an approving review), and we can defer a wider discussion of testing posterior inference to some other time and place.

Perhaps it would be a good topic for an STF team meeting? @dylanhmorris

@dylanhmorris
Copy link
Collaborator

I agree on both counts.

@dylanhmorris
Copy link
Collaborator

Perhaps it would be a good topic for an STF team meeting? @dylanhmorris

Can you open an issue for this in the team materials repo @damonbayer?

@damonbayer damonbayer merged commit 71228e5 into main Jun 20, 2024
6 of 8 checks passed
@damonbayer damonbayer deleted the dmb_cleanup_rtinfections branch June 20, 2024 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants