Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PERF: speed up the adjustment of SNR #36

Closed
wants to merge 4 commits into from
Closed

Conversation

ctrltz
Copy link
Owner

@ctrltz ctrltz commented Oct 1, 2024

The SNR adjustment is currently very slow (13 seconds) when the number of noise sources is 500 (see detailed reports below):

  • 9 seconds are spent for 499 calls of _combine_stcs. In this PR, _combine_sources_into_stc is reworked to combine all sources at once instead of adding them one by one.
  • 3.77 seconds are spent for 6 calls of _get_sensor_space_variance (1 for all noise + 5 for alpha sources). While the calculation of covariance should be the correct approach to account for correlations between sources, in practice it might be computationally easier to actually project the data to sensor space and average the squared values to get mean variance. This strategy is also implemented in this PR.

Current version on master:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   13.749   13.749 /home/willenjoy/Projects/roiextract-simulations/scripts/misc/profile_meegsim.py:11(simulation_ssd_pattern_recovery)
        1    0.000    0.000   13.694   13.694 /home/willenjoy/Projects/meegsim/src/meegsim/simulate.py:187(simulate)
        1    0.001    0.001   13.694   13.694 /home/willenjoy/Projects/meegsim/src/meegsim/simulate.py:245(_simulate)
        1    0.001    0.001   13.297   13.297 /home/willenjoy/Projects/meegsim/src/meegsim/snr.py:104(_adjust_snr)
        1    0.095    0.095    9.526    9.526 /home/willenjoy/Projects/meegsim/src/meegsim/sources.py:173(_combine_sources_into_stc)
      499    0.016    0.000    9.349    0.019 /home/willenjoy/Projects/meegsim/src/meegsim/utils.py:12(combine_stcs)
     1497    4.617    0.003    4.631    0.003 /home/willenjoy/miniforge3/envs/roiextract-simulations/lib/python3.12/site-packages/numpy/lib/function_base.py:5368(insert)
12917/505    0.017    0.000    4.551    0.009 /home/willenjoy/miniforge3/envs/roiextract-simulations/lib/python3.12/copy.py:118(deepcopy)
  529/505    0.004    0.000    4.545    0.009 /home/willenjoy/miniforge3/envs/roiextract-simulations/lib/python3.12/copy.py:247(_reconstruct)
      535    0.004    0.000    4.532    0.008 /home/willenjoy/miniforge3/envs/roiextract-simulations/lib/python3.12/copy.py:217(_deepcopy_dict)
      499    0.001    0.000    4.524    0.009 /home/willenjoy/miniforge3/envs/roiextract-simulations/lib/python3.12/site-packages/mne/source_estimate.py:1099(copy)
     2128    4.502    0.002    4.502    0.002 {method '__deepcopy__' of 'numpy.ndarray' objects}
        6    3.526    0.588    3.770    0.628 /home/willenjoy/Projects/meegsim/src/meegsim/snr.py:9(get_sensor_space_variance)

The same simulation after the optimization:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    1.195    1.195 /home/willenjoy/Projects/roiextract-simulations/scripts/misc/profile_meegsim.py:11(simulation_ssd_pattern_recovery)
        1    0.000    0.000    1.157    1.157 /home/willenjoy/Projects/meegsim/src/meegsim/simulate.py:190(simulate)
        1    0.007    0.007    1.157    1.157 /home/willenjoy/Projects/meegsim/src/meegsim/simulate.py:255(_simulate)
        1    0.008    0.008    0.608    0.608 /home/willenjoy/Projects/meegsim/src/meegsim/snr.py:114(_adjust_snr)
        2    0.000    0.000    0.542    0.271 /home/willenjoy/Projects/meegsim/src/meegsim/source_groups.py:79(simulate)
        2    0.001    0.000    0.542    0.271 /home/willenjoy/Projects/meegsim/src/meegsim/sources.py:130(create)
        6    0.012    0.002    0.518    0.086 /home/willenjoy/Projects/meegsim/src/meegsim/snr.py:10(get_sensor_space_variance)
        2    0.001    0.001    0.514    0.257 /home/willenjoy/Projects/meegsim/src/meegsim/waveform.py:68(one_over_f_noise)

@ctrltz
Copy link
Owner Author

ctrltz commented Oct 15, 2024

Tried to change too many things at once, will split in two PRs.

@ctrltz ctrltz closed this Oct 15, 2024
@ctrltz ctrltz deleted the ctrltz/combine-sources branch October 15, 2024 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant