-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace multiprocessing with joblib for parallelization and change n_cores default to 1 #597
Conversation
Codecov Report
@@ Coverage Diff @@
## main #597 +/- ##
=======================================
Coverage 87.89% 87.89%
=======================================
Files 40 40
Lines 4320 4337 +17
=======================================
+ Hits 3797 3812 +15
- Misses 523 525 +2
Continue to review full report at Codecov.
|
Also, there's an example: https://joblib.readthedocs.io/en/latest/auto_examples/parallel_memmap.html#sphx-glr-auto-examples-parallel-memmap-py |
It was comparing to -1, but should have been using <= 0 when deciding to use all cores.
There's no need for a separate _make_hist method now that we're not applying the method along an axis.
It was using the number of cores for `fit`!
Testing all of these if statements is going to suck...
* Use TQDM to build progress bar for ALESubtraction. * Don't capitalize a comment. * Support parallelization in ALESubtraction. Also, add ALE-difference and logp maps to outputs. * Document changes to SCALE from #597. Whoops! * Fix typo in ALESubtraction docstring. * Add memory_limit parameter to trigger use_memmap. * Check xyz parameter and make it required. * Update tests. * Update examples. * Set memory_limit with a string.
Closes None.
Changes proposed in this pull request:
joblib
instead ofmultiprocessing
for parallelization.joblib
regardless.n_cores
is 1, but I want to be sure before using more complicated code.n_cores
as a parameter.To do:
n_cores
to 1 throughout library.