-
Notifications
You must be signed in to change notification settings - Fork 119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
--benchmark-disable still saves json files #205
Comments
TauPan
added a commit
to TauPan/pytest-benchmark
that referenced
this issue
Jun 11, 2021
Looking at the code this looks like an oversight to me, so I created a PR. |
TauPan
added a commit
to TauPan/pytest-benchmark
that referenced
this issue
Jun 11, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, I just upgraded from 3.2.3 to 3.4.1 (more or less by accident) and noticed that benchmark now saves json files if
--benchmark-disable
is given together with--benchmark-autosave
(I use these flags in pytest.ini so I can just give
--benchmark-enable
and don't need to give autosave as well. Maybe I should not do that, I don't know.)I think the previous behaviour was to issue a warning.
There are a couple of problems I see with this:
1.) I get separate json files for each process I run with pytest-xdist
2.) Those json files contain no actual benchmark information
3.) This throws pytest-watch run with e.g.
--spool 1000 -- --testmon --testmon-forceselect
which will interrupt itself every time the json dump is written. (Of course--ignore=.benchmarks
helps against that.)4.) Those json files clutter the checkout.
My personal opinion is that saving benchmark data makes no sense if no actual benchmarks were run, so I'm happier with the warning in that case.
Any opinion or advice is appreciated here. I'm considering either downgrading, or changing the flags in pytest.ini, since I do rarely run benchmarks. The --ignore flag for pytest only helps against problem number 3.
The text was updated successfully, but these errors were encountered: