Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MRG, MAINT: Fix PySurfer-related crash #7295

Merged
merged 4 commits into from
Feb 10, 2020
Merged

Conversation

larsoner
Copy link
Member

@larsoner larsoner commented Feb 6, 2020

Let's see if larsoner/mayavi@c2a8a23 fixes our Azure failures. Based on the traceback it might.

Will require several restarts of the Azure runs because this is a bit of a random bug.

@larsoner
Copy link
Member Author

larsoner commented Feb 6, 2020

@codecov
Copy link

codecov bot commented Feb 7, 2020

Codecov Report

Merging #7295 into master will decrease coverage by 0.74%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #7295      +/-   ##
==========================================
- Coverage   89.82%   89.07%   -0.75%     
==========================================
  Files         447      447              
  Lines       80694    80974     +280     
  Branches    12876    13032     +156     
==========================================
- Hits        72482    72131     -351     
- Misses       5385     6050     +665     
+ Partials     2827     2793      -34

@larsoner
Copy link
Member Author

larsoner commented Feb 7, 2020

Trying again with a new approach that changes PySurfer's __del__

@larsoner
Copy link
Member Author

larsoner commented Feb 7, 2020

Neither worked :(

Next I'll see if it's a PyVista or Mayavi problem by removing each of those.

PyVista:

@larsoner
Copy link
Member Author

larsoner commented Feb 7, 2020

Pushed a commit to try to clean things up explicitly with gc.collect in case mayavi/pyface/PySurfer/whoever has a bug:

@larsoner
Copy link
Member Author

It looks like this might fix our sporadic failures. I'll go ahead and merge if we get through 10 iterations without error, okay @GuillaumeFavelier @agramfort ?

@GuillaumeFavelier
Copy link
Contributor

It's okay for me

@agramfort
Copy link
Member

yes... although the diff is puzzling...

@larsoner
Copy link
Member Author

yes... although the diff is puzzling...

Briefly, from what I understand:

  • PySurfer does some mlab.close when objects get .close or __del__'ed (i.e., during garbage collection)
  • We can't be sure of when the GC happens (and triggers the __del__)
  • When creating two brain objects, it can be the case that the end up using the same Mayavi figure because of how Mayavi keeps track of its figures and how we create them in PySurfer (this is probably a bug with PySurfer's use of Mayavi)

Thus you can end up in some test creating a Brain instance with a Mayavi figure, then Python does GC and calls the __del__ from a previous Brain that shared that figure, killing the resources, and when your shiny new Brain instance goes to draw somewhere down the line (Mayavi or TVTK) it tries to draw some objects that have been killed by mlab.close.

This PR works around this corner case stuff by forcing GC. Eventually we should probably fix PySurfer, too, but even once we do, these GC calls shouldn't hurt anything.

@agramfort
Copy link
Member

agramfort commented Feb 10, 2020 via email

@larsoner
Copy link
Member Author

This behavior might also now be fixed by nipy/PySurfer#285, but I plan to merge this PR to MNE after test #10 so that we can have stable CIs again while we wait for that to be merged (and released, etc.)

@larsoner larsoner changed the title WIP: Try to fix crash MRG, MAINT: Fix PySurfer-related crash Feb 10, 2020
@larsoner larsoner merged commit 9081ae3 into mne-tools:master Feb 10, 2020
@larsoner larsoner deleted the crash branch February 10, 2020 17:52
@larsoner larsoner mentioned this pull request Feb 19, 2020
AdoNunes pushed a commit to AdoNunes/mne-python that referenced this pull request Apr 6, 2020
* WIP: Try to fix crash

* WIP: Second try

* WIP: Remove PyVista

* FIX: Try another method
AdoNunes pushed a commit to AdoNunes/mne-python that referenced this pull request Apr 6, 2020
* WIP: Try to fix crash

* WIP: Second try

* WIP: Remove PyVista

* FIX: Try another method
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants