Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move dataset objects outside of models #70

Merged
merged 9 commits into from
Mar 25, 2024

Conversation

borauyar
Copy link
Member

Fixes #69 except for the GCNNs.

Datasets are not stored within the model (although this doesn't make an impact on either the speed/memory consumption).
The major memory consuming task was Captum Integrated Gradients, which explodes memory consumption, because many models are generated in memory without batching. Now, we use data loaders and compute feature importance in batches, which caps maximum memory usage to close to the amount used during training.

GNNs' feature importance function still needs to be adapted to this.

@borauyar borauyar linked an issue Mar 25, 2024 that may be closed by this pull request
@borauyar borauyar merged commit cdbea1b into main Mar 25, 2024
@borauyar borauyar deleted the 69-move-dataset-objects-outside-of-models branch December 31, 2024 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move dataset objects outside of models
1 participant