Running multiple test commands in the same directory overwrites coverage

[Blackbaud-ChristiSchneider]

This may be more of a feature request than a bug, not sure.

If you run dotnet test /p:CollectCoverage=true in a directory with multiple test assemblies in it, the coverage output files generated write to the same file so the last assembly to be tested is the only one with coverage reported.

I know that the CoverletOutput property exists, and I was able to have a workaround in this specific case by setting /p:CoverletOutput=coverage-$(cat /dev/random | LC_CTYPE=C tr -dc "[:alpha:]" | head -c 8).json which creates a random file name. This is not ideal (it would be better if it was the project name) and also doesn't solve the case I think is more important.

I'm using Azure Pipelines which use a yaml definition. The .NET Core task allows you to specify multiple projects and to specify arguments, but there is no way I am aware of to access either the project name, an iterator, or a random string within the arguments to set the CoverletOutput to something unique per project.

I tried a couple of things including --configuration $(BuildConfiguration) /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(System.DefaultWorkingDirectory)/TestResults/Coverage/coverage-$(cat /dev/random | LC_CTYPE=C tr -dc "[:alpha:]" | head -c 8).cobertura.xml but this results in an error in my YAML: #Your build pipeline references an undefined variable named ‘cat /dev/random | LC_CTYPE=C tr -dc "[:alpha:]" | head -c 8’. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab. See https://go.microsoft.com/fwlink/?linkid=865972.

[KerbalSpike]

Not sure if this is still relevant to your case, but I've struggled with the same issue for a few days until I realized, that the solution is really simple: just use a relative path for /:pCoverletOutput

Here is the relevant yaml snippet from our pipeline:

steps:
- task: DotNetCoreCLI@2
  displayName: 'Test and collect code coverage'
  inputs:
    command: test
    projects: '**/*.Test.csproj'
    arguments: '--configuration $(BuildConfiguration) --no-build /p:CollectCoverage=true /p:CoverletOutput=TestResults\Coverage\coverage.cobertura.xml /p:CoverletOutputFormat=cobertura  /p:Exclude="[xunit.*]*"'

So this runs dotnet test for all '*.Test.csproj' projects in the solution and generates the coverage output inside the source directories of the test projects.
In essence, simply writing /p:CoverletOutput=cobertura.xml is equivalent to writing /p:CoverletOutput=$(Build.SourcesDirectory)\$(ProjectName)\cobertura.xml (where of course '$(ProjectName)' doesn't actually exist).

The second part to my solution is having reportgenerator collect all those reports using a wildcard pattern:

steps:
- task: Palmmedia.reportgenerator.reportgenerator-build-release-task.reportgenerator@4
  displayName: ReportGenerator
  inputs:
    reports: '$(Build.SourcesDirectory)\*\TestResults\Coverage\coverage.cobertura.xml'
    targetdir: '$(Build.SourcesDirectory)\TestResults\Coverage\Reports'

In theory this works like a charm, however we had to (temporarily) remove coverlet from our pipeline again since we seem to be affected by the "Unable to read beyond end of stream" issue.
#210

[MarcoRossignoli]

@Blackbaud-ChristiSchneider do the @KerbalSpike work around works in your case?

[MarcoRossignoli]

You could also run separate test using /p:MergeWith

For now I'll close for "stale" conversation.
Feel free to re-open if needed!