Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing tests can cause coverage drop when refusing coverage drop #624

Closed
thomas07vt opened this issue Oct 1, 2017 · 0 comments
Closed

Failing tests can cause coverage drop when refusing coverage drop #624

thomas07vt opened this issue Oct 1, 2017 · 0 comments

Comments

@thomas07vt
Copy link
Contributor

Currently simplecov only checks maximum_coverage_drop on a successful test run, so a failing test run can write to .last_run.json with a lower coverage percenage than before.

Here is a scenario:

test coverage is at 100%
configure simplecov to refuse coverage drop, or set max coverage drop to 0
run specs
.last_run.json shows 100%
write some new code
write a spec that only partially covers the new code that fails
run specs
.last_run.json now shows < 100%
fix spec to pass, but test coverage still less that 100%
run specs
.last_run.json still shows < 100%

I have PR that I will raise shortly that fixes this
thomas07vt added a commit to thomas07vt/simplecov that referenced this issue Oct 1, 2017
@thomas07vt
Fixes simplecov-ruby#624 Checks max_coverage_drop on failed tests
7399267
@thomas07vt thomas07vt mentioned this issue Oct 1, 2017
Merged
thomas07vt added a commit to thomas07vt/simplecov that referenced this issue Oct 5, 2017
@thomas07vt
Fixes simplecov-ruby#624 Only write last run result on successes
3d5d338
thomas07vt added a commit to thomas07vt/simplecov that referenced this issue Oct 5, 2017
@thomas07vt
Fixes simplecov-ruby#624 Only write last run result on successes
b7615b3
thomas07vt added a commit to thomas07vt/simplecov that referenced this issue Oct 5, 2017
@thomas07vt
Fixes simplecov-ruby#624 Only write last run result on successes
1141ca3
thomas07vt added a commit to thomas07vt/simplecov that referenced this issue Oct 5, 2017
@thomas07vt
Fixes simplecov-ruby#624 Only write last run result on successes
66ffa82
PragTob added a commit that referenced this issue Oct 6, 2017
@PragTob
Merge pull request #625 from thomas07vt/issues/624/refuse_coverage_dr…
a51ea16 
…op_on_failed_test

Fixes #624 Checks max_coverage_drop on failed tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@thomas07vt