Performance degradation from 3.7.1 to 4.0

[nedbat]

(From DRMacIver) Here's a reproduction for the problem we discussed on IRC.

I've attached a file that shows significant performance loss running under coverage and modest performance loss under 4.0a5 vs 3.7.1

These are run against 32a214a141bbc81bd624890216899261745f8588 (bb) of [email protected]:DRMacIver/hypothesis.git. Note that if you run against 1.2.1 this test has significantly different performance characteristics (it will be much faster).

Command I used to run, from the root of a hypothesis checkout:

With coverage: time PYTHONPATH=src python -m coverage run --branch --rcfile=/dev/null test_coverage_perf.py

Without coverage: time PYTHONPATH=src python test_coverage_perf.py

Numbers are about as variable as you would expect from this extremely shonky benchmarking method, but roughly what I'm seeing is:

Without coverage: ~14s
With coverage 3.7.1: ~35s
With coverage 4.0a5: ~50s

This is running on CPython 3.3.6 on an oldish laptop running Linux Mint. Coverage was using CTracer in both instances.

Let me know if there's anything I can do to help with debugging this or if you need more information.

---

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/387
• This issue had attachments: test_coverage_perf.py. See the original issue for details.

[nedbat]

Thanks for the test case. I've reproduced your results. Normalizing the non-coverage time to 1.0, running under coverage 3.7.1 is 2.28, and under coverage 4.0a6 is 2.99, for a 31% slowdown between coverage versions.

I also ran under 2.7.10, which had a similar-ish slowdown between coverage versions. The surprising thing was the non-coverage run time for your code was 4.3x slower under Python 3.3.6 than under 2.7.10!?

[nedbat]

Moving the FileDisposition implementation into C so that attribute access is fast fixed this. Final work in 154e0a81e945 (bb).