Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refac: Move N2V masking in lightning module #365

Open
wants to merge 49 commits into
base: main
Choose a base branch
from

Conversation

CatEek
Copy link
Contributor

@CatEek CatEek commented Jan 21, 2025

  • What: Rewrote all N2V related transforms in torch and moved them to lightning module. It's now applied over batch dimension. Seems like it's also faster now
  • Why: Them being in the dataset created unnecessary complication for the dataset classes and problem integrating LVAE based algos
  • How: With my crabby claws, unfortunately

Changes Made

  • Added: N2VMAnipulate is now in the preprocess param of the lightning module. It is a function that is applied to a batch before inputting it to the model.
  • Modified: Many files
  • Removed: Some old tests

Additional Notes and Examples

  • Only tested N2V performance on BSD, still need to test others.
  • There're a bunch of todos related to some ugly hack here and there. Needs to be addressed later
  • Need to update docs

Please ensure your PR meets the following requirements:

  • Code builds and passes tests locally, including doctests
  • New tests have been added (for bug fixes/features)
  • Pre-commit passes
  • PR to the documentation exists (for bug fixes / features)

@jdeschamps
Copy link
Member

There are missing docs in the torch N2V manipulate: https://results.pre-commit.ci/run/github/607105530/1737501613.FBC6iWDHTgWR3WhJeX4o9A

src/careamics/config/__init__.py Outdated Show resolved Hide resolved
if self.algorithm == "n2v":
x_preprocessed, *aux = self.preprocess(x)
else:
x_preprocessed = x
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In self.preprocess, non-N2V algorithm returns Identity. Why is it not used here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, then can remove the "if self.algorithm == "n2v""

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because in other cases input to the Identity can be a list, which won't work. This whole construction is quite ugly, and definitely needs revisiting.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just tested it and I get

>>> torch.nn.Identity()([torch.tensor([1,2,3])])
[tensor([1, 2, 3])]

This is the case where targets are a list?

I've also noticed this function is typed incorrectly batch should have the type list[torch.Tensor] I believe.

src/careamics/lightning/lightning_module.py Show resolved Hide resolved
src/careamics/lightning/lightning_module.py Show resolved Hide resolved
src/careamics/transforms/pixel_manipulation_torch.py Outdated Show resolved Hide resolved
tests/config/test_n2v_preprocessing.py Show resolved Hide resolved
tests/test_careamist.py Show resolved Hide resolved
Copy link
Member

@melisande-c melisande-c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't looked at everything fully, left a few comments

src/careamics/config/algorithms/n2v_algorithm_model.py Outdated Show resolved Hide resolved
src/careamics/config/algorithms/n2v_algorithm_model.py Outdated Show resolved Hide resolved
src/careamics/transforms/compose.py Show resolved Hide resolved
src/careamics/dataset/in_memory_pred_dataset.py Outdated Show resolved Hide resolved
src/careamics/lightning/lightning_module.py Show resolved Hide resolved
if self.algorithm == "n2v":
x_preprocessed, *aux = self.preprocess(x)
else:
x_preprocessed = x
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, then can remove the "if self.algorithm == "n2v""

src/careamics/lightning/lightning_module.py Show resolved Hide resolved
@CatEek
Copy link
Contributor Author

CatEek commented Jan 23, 2025

@melisande-c @jdeschamps @veegalinova Thanks for the comments, I'll take a look. Another thing is there's probably some inconsistencies in the configuration factories. Like I'm not sure N2V transform should be defined there

@jdeschamps jdeschamps changed the title Move N2V manipulation out of transforms list refac: Move N2V manipulation in lightning module Jan 29, 2025
@jdeschamps jdeschamps changed the title refac: Move N2V manipulation in lightning module refac: Move N2V masking in lightning module Jan 29, 2025
This was referenced Jan 23, 2025
@jdeschamps
Copy link
Member

jdeschamps commented Feb 12, 2025

A summary of the latest changes since the reviews:

There are some stuff that will need refactoring or splitting into multiple functions, but better open issues and deal with them later at this point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants