Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create code coverage reports #657

Merged
merged 30 commits into from
Jul 20, 2019
Merged

Create code coverage reports #657

merged 30 commits into from
Jul 20, 2019

Conversation

angularsen
Copy link
Owner

@angularsen angularsen commented Apr 21, 2019

Testing out codecov.io for test coverage reports on pull requests, with diffs against master.

I tried OpenCoverage first, but I ran into heaps of trouble and it seems the project is not very actively maintained. dotCover CLI tools does not require a license as far as I can tell and is much faster to run.

@angularsen angularsen changed the title Run opencover in AppVeyor (WIP) Add code coverage (WIP) Apr 21, 2019
@angularsen angularsen changed the title Add code coverage (WIP) Create code coverage reports (WIP) Apr 21, 2019
@angularsen
Copy link
Owner Author

angularsen commented Apr 21, 2019

Currently getting error messages about not finding .pdb files on AppVeyor, but the same commands work locally when building with build.bat. Not sure why exactly.

https://ci.appveyor.com/project/angularsen/unitsnet/builds/24002213

OpenCover.Console.exe -target:"dotnet.exe" -targetargs:"test UnitsNet.Tests" -output:"Artifacts/UnitsNet.Tests.coverage.xml" -filter:"+[UnitsNet*]* -[UnitsNet.Tests*]*" -oldstyle -register:user
Executing: C:\Program Files\dotnet\dotnet.exe
Test run for C:\projects\unitsnet\Artifacts\UnitsNet.Tests\netcoreapp2.0\UnitsNet.Tests.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 16.0.1
Copyright (c) Microsoft Corporation.  All rights reserved.
Starting test execution, please wait...
Total tests: 2736. Passed: 2736. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 11.3785 Seconds
Committing...
[NOTE] Post processing coverage data...
No results, this could be for a number of reasons. The most common reasons are:
    1) missing PDBs for the assemblies that match the filter please review the
    output file and refer to the Usage guide (Usage.rtf) about filters.
    2) the profiler may not be registered correctly, please refer to the Usage
    guide and the -register switch.

Links:
https://automationrhapsody.com/code-coverage-net-core-unit-tests-opencover/
https://www.appveyor.com/blog/2017/03/17/codecov/

@codecov-io
Copy link

codecov-io commented Jul 18, 2019

Codecov Report

❗ No coverage uploaded for pull request base (master@1c6c629). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##             master     #657   +/-   ##
=========================================
  Coverage          ?   55.56%           
=========================================
  Files             ?      164           
  Lines             ?    37408           
  Branches          ?        0           
=========================================
  Hits              ?    20786           
  Misses            ?    16622           
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c6c629...808808f. Read the comment docs.

@angularsen angularsen changed the title Create code coverage reports (WIP) Create code coverage reports Jul 18, 2019
@angularsen
Copy link
Owner Author

Well whoop, it finally works. The numbers are horrible of course, but we'll improve that by either filtering out stuff or converting tests to test for N units.

@angularsen angularsen requested a review from tmilnthorp July 18, 2019 00:28
@tmilnthorp
Copy link
Collaborator

Looks good - but yikes. Do you mean that coverage isn't calculated properly until we switch to NUnit from XUnit?

@angularsen
Copy link
Owner Author

angularsen commented Jul 18, 2019

I didn't mean NUnit :-D

I meant going from "hey here is a test case that tests Length, Mass and some other widely used quantities all in one go" to maybe generating unique test cases for every quantity and unit - in order to get better test coverage.

But yeah, I'll have to look closer at the numbers. I was mostly happy that I got it working at all as a start :-P

@angularsen
Copy link
Owner Author

I'll take a second look at why the numbers are so bad before merging this.

@angularsen
Copy link
Owner Author

@tmilnthorp Wohoo! After switching from OpenCover to dotCover I got more sane results and way better performance. 56% coverage now, which sounds reasonable given that many tests only target specific quantities.

I believe the licensing for the CLI tooling is OK, but I have also applied for an open source license just to be sure. Merging this now so we can get a baseline going for future PRs.

@angularsen angularsen merged commit 9c12fb1 into master Jul 20, 2019
@angularsen angularsen deleted the new/codecov branch July 20, 2019 14:58
@tmilnthorp
Copy link
Collaborator

Sweet. Still surprised though given all the As/To methods

@angularsen angularsen mentioned this pull request Jul 20, 2019
@angularsen
Copy link
Owner Author

Added issue to follow up improving the coverage numbers:
#682

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants