Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace multiprocessing with joblib for parallelization and change n_cores default to 1 #597

Merged
merged 26 commits into from
Feb 13, 2022

Conversation

tsalo
Copy link
Member

@tsalo tsalo commented Nov 5, 2021

Closes None.

Changes proposed in this pull request:

To do:

@codecov
Copy link

codecov bot commented Nov 5, 2021

Codecov Report

Merging #597 (fedd3a8) into main (c35ca9b) will increase coverage by 0.00%.
The diff coverage is 93.45%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #597   +/-   ##
=======================================
  Coverage   87.89%   87.89%           
=======================================
  Files          40       40           
  Lines        4320     4337   +17     
=======================================
+ Hits         3797     3812   +15     
- Misses        523      525    +2     
Impacted Files Coverage Δ
nimare/workflows/ale.py 92.42% <ø> (ø)
nimare/workflows/macm.py 25.71% <ø> (ø)
nimare/workflows/scale.py 80.55% <ø> (ø)
nimare/meta/cbma/ale.py 92.74% <84.21%> (-1.41%) ⬇️
nimare/utils.py 95.41% <95.45%> (-0.06%) ⬇️
nimare/base.py 88.18% <100.00%> (ø)
nimare/diagnostics.py 95.29% <100.00%> (-0.11%) ⬇️
nimare/meta/cbma/base.py 94.69% <100.00%> (+0.55%) ⬆️
nimare/meta/cbma/mkda.py 96.31% <100.00%> (ø)
nimare/meta/ibma.py 100.00% <100.00%> (ø)
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c35ca9b...fedd3a8. Read the comment docs.

@tsalo tsalo marked this pull request as draft November 5, 2021 18:17
@tsalo tsalo changed the title [REF] Use partial functions and remove if statements for MP calls [REF] Replace multiprocessing with Joblib Nov 6, 2021
@tsalo tsalo added the refactoring Requesting changes to the code which do not impact behavior label Nov 6, 2021
@tsalo tsalo changed the title [REF] Replace multiprocessing with Joblib Replace multiprocessing with joblib for parallelization Dec 14, 2021
@tsalo tsalo marked this pull request as ready for review February 10, 2022 14:42
@tsalo tsalo added the breaking-change PRs that change results or interfaces. label Feb 10, 2022
@tsalo tsalo merged commit 86eda1c into neurostuff:main Feb 13, 2022
@tsalo tsalo changed the title Replace multiprocessing with joblib for parallelization Replace multiprocessing with joblib for parallelization and change n_cores default to 1 Feb 13, 2022
tsalo added a commit to tsalo/NiMARE that referenced this pull request Feb 16, 2022
tsalo added a commit that referenced this pull request Feb 17, 2022
* Use TQDM to build progress bar for ALESubtraction.

* Don't capitalize a comment.

* Support parallelization in ALESubtraction.

Also, add ALE-difference and logp maps to outputs.

* Document changes to SCALE from #597.

Whoops!

* Fix typo in ALESubtraction docstring.

* Add memory_limit parameter to trigger use_memmap.

* Check xyz parameter and make it required.

* Update tests.

* Update examples.

* Set memory_limit with a string.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change PRs that change results or interfaces. refactoring Requesting changes to the code which do not impact behavior
Projects
Development

Successfully merging this pull request may close these issues.

1 participant