Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: switch to codecov CircleCI orb #995

Merged
merged 6 commits into from
May 13, 2022

Conversation

Eric-Warehime
Copy link
Contributor

Summary

Add a codecov config file to directly configure upload steps instead of using the bash uploader since that has been deprecated.

@Eric-Warehime Eric-Warehime changed the title Initial codecov upload update CI: Initial codecov upload update May 10, 2022
@Eric-Warehime Eric-Warehime added the Enhancement New feature or request label May 10, 2022
@Eric-Warehime Eric-Warehime force-pushed the codecov-update branch 2 times, most recently from fbbca8c to ed5a6c9 Compare May 11, 2022 00:01
@codecov-commenter
Copy link

codecov-commenter commented May 11, 2022

Codecov Report

Merging #995 (7c5243c) into develop (491c684) will decrease coverage by 1.28%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop     #995      +/-   ##
===========================================
- Coverage    60.43%   59.15%   -1.29%     
===========================================
  Files           42       45       +3     
  Lines         5750     8353    +2603     
===========================================
+ Hits          3475     4941    +1466     
- Misses        1827     2950    +1123     
- Partials       448      462      +14     
Impacted Files Coverage Δ
util/util.go 70.00% <0.00%> (-10.77%) ⬇️
accounting/eval_preload.go 77.24% <0.00%> (-3.89%) ⬇️
api/handlers.go 72.68% <0.00%> (-1.63%) ⬇️
api/generated/v2/routes.go 17.76% <0.00%> (-1.17%) ⬇️
fetcher/fetcher.go 33.77% <0.00%> (-0.96%) ⬇️
api/converter_utils.go 91.23% <0.00%> (-0.96%) ⬇️
idb/postgres/internal/util/util.go 73.52% <0.00%> (-0.94%) ⬇️
processor/blockprocessor/block_processor.go 83.33% <0.00%> (-0.88%) ⬇️
idb/postgres/internal/writer/write_txn.go 82.67% <0.00%> (-0.83%) ⬇️
...gres/internal/migrations/convert_account_data/m.go 62.58% <0.00%> (-0.57%) ⬇️
... and 35 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 491c684...7c5243c. Read the comment docs.

@algojack
Copy link
Contributor

I know you are still working on this, but once you are ready if you can please replace upload_coverage on line 64 in .circleci/config.yml too, and delete the upload_coverage.sh script

@Eric-Warehime
Copy link
Contributor Author

I looked into why the coverage is changing in this PR (causing it to fail as intended when coverage decreases) even though we're not actually touching the code being tested.

I suspect the answer is hidden in this bash script though I haven't identified the exact cause. It is re-writing the coverage file before uploading (you can see the # path=fixes section in a coverage file produced by the bash script like this one which does not exist in the uploaded file from the new codecov uploader. This leads me to believe there is some altering of data happening affecting which files we're tracking.

Regardless of the root cause of the above, the difference we're seeing is that the previously untracked files config/config.go, util/metrics/metrics.go, and util/diff.go are now being reported in new coverage reports which is causing the coverage to drop. I think we can either ignore that failure and merge this or I can update the codecov config to not fail on coverage drops in this PR and submit a followup PR to enable the failure which should show a 0% drop and pass.

@Eric-Warehime Eric-Warehime marked this pull request as ready for review May 13, 2022 00:00
@winder
Copy link
Contributor

winder commented May 13, 2022

What do you think about setting a coverage target for new code only? We can discuss the target, I'd consider something like 75% to start but maybe we can go higher?

https://docs.codecov.com/docs/common-recipe-list#ensure-all-code-is-covered

@Eric-Warehime
Copy link
Contributor Author

The default config has a project check which checks the coverage of the project as a whole is non-decreasing, and a patch check which verifies the coverage in the PR is greater than some limit you set.

I disabled the patch in favor of just ensuring that we're always increasing coverage as a whole. My thinking was that setting 70% per patch will never guarantee that we get above 70% while just ensuring that we're improving the project's coverage in theory will.

I'm open to different settings if people have strong opinions about it.

@Eric-Warehime
Copy link
Contributor Author

Discussed this morning and we've agreed to track the coverage % in the patch, and to initially prevent it from blocking PRs on failure. In the future as we get more complete coverage we will enable it to block PRs on failure.

@winder winder changed the title CI: Initial codecov upload update CI: switch to codecov github action May 13, 2022
@winder winder merged commit 7dbf4b4 into algorand:develop May 13, 2022
@cce cce changed the title CI: switch to codecov github action CI: switch to codecov CircleCI orb Jun 23, 2022
@cce
Copy link
Contributor

cce commented Jun 23, 2022

updated title, this uses a circleci orb, not a github action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request Team Lamprey
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants