Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make manim crash when ffmpeg crashes #546

Merged
merged 13 commits into from
Oct 16, 2020
Merged

Make manim crash when ffmpeg crashes #546

merged 13 commits into from
Oct 16, 2020

Conversation

leotrs
Copy link
Contributor

@leotrs leotrs commented Oct 15, 2020

Motivation

Sometimes, if FFMPEG crashes, manim happily keeps chugging along without reporting anything. It even prints a message as if everything was ok. In the picture below, FFMPEG complains that it didn't find a necessary file, but manim reports that the file is ready to view. In fact, manim didn't render anything at all!

2020-10-14-213923_1126x151_scrot

Importantly, this allows some tests to pass even if FFMPEG has NOT rendered anything!

Explanation for Changes

Instead of calling FFMPEG and just waiting for it to finish, use subprocess.check_call, which will raise an exception unless the subprocess terminates successfully.

Testing Status

Tests pass locally.

Acknowledgement

@leotrs leotrs added pr:bugfix Bug fix for use in PRs solving a specific issue:bug enhancement Additions and improvements in general pr:deprecation Deprecation, or removal of deprecated code pr:easy review There is nothing particular (i.e, it's about a general/small thing) to know for review! labels Oct 15, 2020
@leotrs leotrs self-assigned this Oct 15, 2020
@eulertour
Copy link
Member

Building the docs fails for me locally too 😕
Which sort of makes sense, because although the docs do build on master they output errors like this a few times:
manim/docs/source/media/videos/480p30/partial_movie_files/Example3DNo1-1/partial_movie_file_list.txt: Invalid data found when processing input

I think manim is trying to do something with partial movie files even when it's been instructed to only save the last frame, meaning there are no partial movie files to process in the first place.

@huguesdevimeux
Copy link
Member

This looks all good, but can you do this for the other ffmpeg commands as well ?

@kolibril13
Copy link
Member

Looks good to me as well.
However I wonder why the docs build saies:
'/home/docs/checkouts/readthedocs.org/user_builds/manimce/checkouts/546/docs/source/media/videos/480p30/Example3DNo1-1.mp4']' returned non-zero exit status 1.
The strange thing is, that there is only Example3DNo1 and no Example3DNo1-1.
I will make a commit and temporary remove Example3DNo1, so maybe this will solve this.

@behackl
Copy link
Member

behackl commented Oct 15, 2020

Looks good to me as well.
However I wonder why the docs build saies:
'/home/docs/checkouts/readthedocs.org/user_builds/manimce/checkouts/546/docs/source/media/videos/480p30/Example3DNo1-1.mp4']' returned non-zero exit status 1.
The strange thing is, that there is only Example3DNo1 and no Example3DNo1-1.
I will make a commit and temporary remove Example3DNo1, so maybe this will solve this.

@kolibril13 No! Remember: the documentation slightly changes file names in order to make them unique; if you remove this one example, I am rather sure another one will fail.

Please revert your commit again. I fear that we need to find a tailored solution for the documentation, or we will need to fix the underlying issue first...

@kolibril13
Copy link
Member

Ok, I will read this example, now Example3DLightSourcePosition is breaking.
Its interesting that first of all the 3d scenes seems to be affectd

@behackl
Copy link
Member

behackl commented Oct 15, 2020

Because the files in 3d_scenes are rendered first.

@kolibril13
Copy link
Member

Because the files in 3d_scenes are rendered first.

Ok, I am only hoping for an easy fix :)
I will once run it without the 3d scenes and read them as soon as the docs run should fail.

@behackl
Copy link
Member

behackl commented Oct 15, 2020

You can check out one of the last successful builds (https://readthedocs.org/projects/manimce/builds/12105792/) and search for "invalid data found" in the last log field. The error occurs 30 times.

@leotrs
Copy link
Contributor Author

leotrs commented Oct 15, 2020

Aha! I think I may have inadvertently uncovered the very bug that sent me down this path in the first place. Imma fix the docs and report back.

@leotrs
Copy link
Contributor Author

leotrs commented Oct 16, 2020

All checks green now. This was super hard to debug, mainly because of the mess of the config system and scene file writer.

The tests pass on CI, but not locally, which is weird. Locally, I'm seeing problems with tests/test_pango.py, so think I may just have screwed up my setup. Can anyone please checkout this branch and run the tests locally?

Copy link
Member

@behackl behackl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes LGTM, the pipeline passes and the examples I checked sporadically in the rendered documentation also seem okay.

Thanks for your efforts!

@behackl behackl merged commit 8b9dba2 into master Oct 16, 2020
@behackl behackl deleted the subprocess-check-call branch October 16, 2020 23:56
@eulertour eulertour mentioned this pull request Oct 17, 2020
eulertour added a commit that referenced this pull request Oct 17, 2020
eulertour added a commit that referenced this pull request Oct 17, 2020
@leotrs
Copy link
Contributor Author

leotrs commented Oct 17, 2020

Ok this broke the --disable_caching flag (thanks @eulertour for realizing and reverting).

In order to keep working on this, I want to implement tests first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Additions and improvements in general pr:bugfix Bug fix for use in PRs solving a specific issue:bug pr:deprecation Deprecation, or removal of deprecated code pr:easy review There is nothing particular (i.e, it's about a general/small thing) to know for review!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants