Publishing symbols with releases

[jnm2]

To repeat the blurb from nunit/nunit#2421:

This will affect the team's day jobs and side projects since it will be easy to drop into NUnit code any time you want to understand what's going on. It will also affect NUnit users who may return the courtesy by taking some initiative in diagnosing bugs they observe. Creating a repro is much easier when you aren't dealing with a black box!

While wanting to step into adapter code might be less common than wanting to step into the framework,
it's still something that would benefit those of us who do.

What do you think of packaging either a GitLinked or source-linked PDB for the adapter assembly into the VSIX and NuGet packages?

I'm happy to do (a few) size comparisons like I did at nunit/nunit#2421 (comment). We went with GitLink for the framework which increased the nupkg size to 2.51× the original.

[jnm2]

@OsirisTerje How do you feel about this one and 3.10? NUnit Framework 3.10 is adding it, so there could be a nice symmetry, and we get to debug against it that much sooner. If you have any concerns about it, it can certainly wait.

[OsirisTerje]

@jnm2 What is the size impact? That is my concern.

[rprouse]

The NUnit Framework NuGet increased from 1.5 MB to 3.75 MB if the numbers in the issue are correct.

[jnm2]

Wow, this is way better than the NUnit package!

Options	nupkg	vsix	zip	Min VS
No PDBs	402 KB (1.00×)	224 KB (1.00×)	3,232 KB
Portable PDBs + embed source	515 KB (1.28×)	280 KB (1.25×)	3,335 KB	2017.5
Portable PDBs + SourceLink	421 KB (1.05×)	234 KB (1.04×)	3,242 KB	2017
Windows PDBs + GitLink	484 KB (1.20×)	266 KB (1.19×)	3,306 KB	2005

At those great ratios, the selling point of embedded source is that it works without network access and having to ask for permission. Downside is that only VS 2017.5 and newer are aware of it.

The selling point of GitLink is that VS2005 and newer are aware of it.

Should I do a PR to add GitLink, which will add 13% to the current size?

Also, I added the results for the VSIX. So long as we keep shipping the VSIX, I don't know why we wouldn't treat it the same. I would have already found VSIX PDBs useful.

[CharliePoole]

IIRC we were starting to move away from VSIX.

[OsirisTerje]

The VSIX is still pretty popular, the stats are as shown below:

60K in 90 days is a lot, and a high view/download ratio too. And there is no downwards trend for the last 90 days. I don't think we need to include any pdb/source information with the vsix, but we should try to keep it in the release until we see a clear phase-out of users.

[jnm2]

Oops, didn't see that dotnet publish was rebuilding and overwriting with a portable PDB. Numbers in GitLink row were wrong. Updated the table with double-checked numbers in all rows.

I know Brad has dropped the xUnit VSIX, so it wouldn't be terrible if we followed his lead there. If we do remove the VSIX in 3.10, the order in which that issue and this one would be implemented probably wouldn't matter a whole lot.

I'm involved in a number of .NET Framework projects, and for those you don't need to reference the VSTest SDK and the VSTest adapter from each project. I've always disliked adding them in .NET Framework projects because I considered that an implementation detail of build.cake, if you were even tied to VSTest in the first place (I prefer NUnit console). And the adapter VSIX is the best place for an implementation detail of the IDE Test Explorer window; as a ReSharper and NCrunch user, the adapter isn't even necessary there.

[jnm2]

@OsirisTerje I may be remembering wrong but I seem to remember wanting to step into the VSIX source. However, if you see value in not including the PDB in the VSIX, that's fine and I'll come back and start complaining later if I really need it. The only other thing is that we will have to draw that distinction in the docs so that VSIX users who read the docs don't waste time trying to source-step.

[OsirisTerje]

I don't have any strong opinions on it, just being careful of it not being too big, but thinking about it, size is probably less important with the vsix than the nuget since it is not downloaded for every build and every solution. So, perhaps better to just include it.

About VSIX, IIRC NUnit is now the only adapter with a vsix version, and seeing the download numbers I feel we should continue with it until 1) users are dropping off significantly 2) something technical prevents us from updating it further.
We know, and should make that even more clear on the marketplace page, that it doesn't run .net core , and we should add some information on that page about the nuget adapter package, to make users aware that they can move over. It might be that some people simply don't know.

[jnm2]

To put the numbers above in perspective, the nupkg was 486 KB and the VSIX was 298 KB prior to the Mono.Cecil removal. That's 1.2× and 1.3× what it is now.

[jnm2]

@OsirisTerje That seems like a good plan for the VSIX documentation.

Which way are you leaning—GitLink like the framework, or SourceLink or embedded source?

[OsirisTerje]

No strong opinion, but as small as possible, which seems to indicate sourcelink, but frankly, if you used gitlink with the framework and are happy with that, the adapter should follow suit. Embedded source is bigger, and I don't think we should go that way. This is not something everyone would use anyway, right?
@rprouse What do you think?

[jnm2]

SourceLink is the least fiddly besides embedded source. If source-stepping in VS2015 and earlier is not important to you, I'd say SourceLink for sure.