Coverlet Intergration with VSTest

[vagisha-nidhi]

Integration with VSTest
Implementation details and user documentation here : #402

Overview of Implementaion :

1. Ouproc Datacollector : coverlet.collector/DataCollector/CoverletCoverageDataCollector.cs
   a. Instrumenting modules in OnSessionStart of CoverletCoverageDataCollector
   b. Collecting coverage report in OnSessionEnd of CoverletCoverageDataCollector
   c. Sending coverage report back to test platform in OnSessionEnd
2. Inproc Datacollector : coverlet.inproccollector/CoverletInProcDataCollector.cs, as contributed in [WIP] Implement a VSTest data collector to write coverage data to disk #329

contributes to #395

[MarcoRossignoli]

We support .NET Full...but not multiple tests modules AFAIK cc: @tonerdo

[MarcoRossignoli]

I did a part of review(until OnSessionEnd out of proc collector), I'll be OOF for fast holiday for the rest of the week, I'll go on from sunday!

[tonerdo]

A few comments

Wrong review summary

[tonerdo]

LGTM! Module two comments

[tonerdo]

@vagisha-nidhi this is good to go, thank you! Out of curiosity, will this just work or is there an accompanying change that needs to be done in vstest or otherwise?

[vagisha-nidhi]

@tonerdo Yeah. There have been changes in vstest which we have made but waiting on https://github.com/dotnet/cli to release with the latest changes. This might take a week. I will update you with that. We should be able to release the nuget once that is done.

[tonerdo]

@vagisha-nidhi ok thanks. Can you please point me to the changes in the two repos, for my learning benefit

[vagisha-nidhi]

@tonerdo Sure.
TestPlatform : microsoft/vstest#1962, microsoft/vstest#1969, microsoft/vstest#2015, microsoft/vstest#1991

Dotnet cli : These changes are going in dotnet cli in 16.1.0 release of test platform. TP insertion here dotnet/cli#11312

[MarcoRossignoli]

I like it!

[MarcoRossignoli]

LGTM thank's @vagisha-nidhi!

[MarcoRossignoli]

@tonerdo we can merge!

[MarcoRossignoli]

@vagisha-nidhi I would like to test locally but I think we need to wait for VSTests PRs?

[vagisha-nidhi]

@vagisha-nidhi I would like to test locally but I think we need to wait for VSTests PRs?

@MarcoRossignoli You can test locally. Get the dlls from https://www.nuget.org/packages/Microsoft.TestPlatform/ version 16.1.0. You can replace these dlls in dotnet sdk folder and run dotnet test --collect:"Xplat code coverage" on a test project.
Also, generate coverlet.collector nuget by running dotnet msbuild .\build.proj /p:Configuration=Release in coverlet repo and add <PackageReference Include="coverlet.collector" Version="1.0.0" /> to the test project csproj

Please let me know if you face any issues.

[vagisha-nidhi]

@vagisha-nidhi I would like to test locally but I think we need to wait for VSTests PRs?

@MarcoRossignoli You can test locally. Get the dlls from https://www.nuget.org/packages/Microsoft.TestPlatform/ version 16.1.0. You can replace these dlls in dotnet sdk folder and run dotnet test --collect:"Xplat code coverage" on a test project.
Also, generate coverlet.collector nuget by running dotnet msbuild .\build.proj /p:Configuration=Release in coverlet repo and add <PackageReference Include="coverlet.collector" Version="1.0.0" /> to the test project csproj

Please let me know if you face any issues.

@MarcoRossignoli This might not work I just checked. Else, you can pull the latest master from vstest, build it, and you will find netcore dlls in artifacts/Debug/netcoreapp2.0 folder. Replace these in dotnet sdk.

[MarcoRossignoli]

@MarcoRossignoli This might not work I just checked. Else, you can pull the latest master from vstest, build it, and you will find netcore dlls in artifacts/Debug/netcoreapp2.0 folder. Replace these in dotnet sdk.

Ok I'll do some test on my local with fresh sdk.

[MarcoRossignoli]

@vagisha-nidhi I tried but i get exception I copied artifacts/Debug/netcoreapp2.0 to dotnet\sdk\3.0.100-preview6-012013

Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'Microsoft.DotNet.PlatformAbstractions, Version=2.2.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The located assembly's manifest definition does not match the assembly reference. (0x80131040)
   at Microsoft.DotNet.Cli.MulticoreJitProfilePathCalculator.CalculateProfileRootPath()
   at Microsoft.DotNet.Cli.MulticoreJitActivator.StartCliProfileOptimization()
   at Microsoft.DotNet.Cli.MulticoreJitActivator.TryActivateMulticoreJit()
   at Microsoft.DotNet.Cli.Program.Main(String[] args)

[vagisha-nidhi]

@vagisha-nidhi I tried but i get exception I copied artifacts/Debug/netcoreapp2.0 to dotnet\sdk\3.0.100-preview6-012013

Unhandled Exception: System.IO.FileLoadException: Could not load file or assembly 'Microsoft.DotNet.PlatformAbstractions, Version=2.2.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. The located assembly's manifest definition does not match the assembly reference. (0x80131040)
   at Microsoft.DotNet.Cli.MulticoreJitProfilePathCalculator.CalculateProfileRootPath()
   at Microsoft.DotNet.Cli.MulticoreJitActivator.StartCliProfileOptimization()
   at Microsoft.DotNet.Cli.MulticoreJitActivator.TryActivateMulticoreJit()
   at Microsoft.DotNet.Cli.Program.Main(String[] args)

@MarcoRossignoli Copy everything except Microsoft.DotNet.PlatformAbstractions and Microsoft.Extensions.DependencyModel. This error has something to do with version mismatch of the assembly, which are not part of vstest.

[MarcoRossignoli]

@vagisha-nidhi another unrelated question...is there a way inside VS to "Set test as startup" and launch debug with F5?
I can do it using ctrl+r+t but I need to be on "test to run"...it's very uncomfortable, I would like setup startup test, put a breakpoint on code and forget about test position...I used to write custom xunit runner and passing -method like corefx repo...it's very fast.

[vagisha-nidhi]

breakpoint

I am not sure if such an option is available.