Switch symbols output to snupkg format

[natemcmaster]

The .snupgk format is more efficient than the .symbols.nupkg format because it only includes .pdbs. It doesn't also include the .dll files and source code, which are unnecessary in most usages due to source-link and the way .symbols.nupkg files are actually handled by our CI.

Changing formats should be a matter adding <SymbolPackageFormat>snupkg</SymbolPackageFormat> to Directory.Build.props and ensuring that the .snupkg files end up in the right locations (blob storage, drop shares, symbols servers, etc.)

[natemcmaster]

Triage team: let's discuss first with Arcade about how we should handle symbols.

[JunTaoLuo]

@dotnet/dnceng Is there a concerted effort to do this across all teams?

[mmitche]

/cc @tmat @markwilkie I don't think there is one yet, but getting it on the schedule would be good.

[michellemcdaniel]

We will also need to update all of our symbols/sourcelink validation scripts at the same time, so that we look for the right file types.

[riarenas]

One consideration is that some internal Microsoft tooling require the dlls to be submitted to the symbol servers for indexing. If the new format does not include the dlls, we'll have to tweak the publishing infra to keep submitting them.

[dougbu]

External and blocked until Arcade does the needful

[dougbu]

@markwilkie @riarenas are there any plans to do the needful in Arcade❔ If not, we should close this issue…

[markwilkie]

Can someone describe the value again for me? Efficiency? Is there a customer scenario that's currently being missed?

[dougbu]

I don't remember the whole history but believe .snupkg packages were smaller, perhaps not containing the assemblies as well as the symbol files❔ @aortiz-msft @DoRonMotter was this a NuGet push and is it still the direction we should be moving❔

[mmitche]

I believe the .snupkgs are also pushed to the nuget server, in addition to the normal symbol servers. Note that this appears to be only applicable to nuget.org servers.

https://docs.microsoft.com/en-us/nuget/create-packages/symbol-packages-snupkg

I think we should identify whether there is an experience gap for our customers before we go off and implement this. @hoyosjs who might be an expert in this area?

[hoyosjs]

snupkg is an advantage in the sense that they are indexed to NuGet and that they are lighter. There are a few problems with them right now for our use:

• They don't support full Windows PDBS - so they don't support native code, only managed portable PDBS.
• We need DLLs indexed into our symbol servers. From a debugging perspective, sourcelink-enabled PDBs with embedded untracked source do the work for debugging, but this is just source debugging. For dump debugging, DLLs are often needed as they contain metadata that's needed but that is often not available in the dumps (comming from desire to avoid collecting full dumps for file size reasons and some PII protected scenarios where most info is stripped). This is crucial for things like Watson and SOS. Currently, as @riarenas pointed out, V3 publishing only tries to publish the symbol packages so this would need to change.

Microsoft Symbol servers (symweb and MSDL) index all of this for us and they have public accessibility as needed. NuGet symbol servers make a lot of sense if you are an external customer/OSS project that wants to index symbols. Unless I am missing something the only thing they provide for us is cost savings on the per-repo payload size. If this is a concern, or if there's a customer scenario that I am missing we should consider this. Otherwise, I am not sure this gives us much of a benefit and requires quite a few changes in the overall dotnet infra. I know #dotnet/arcade#1959 has some recent feedback, so maybe @ViktorHofer and @tmat can chime in if they have any further info.

[markwilkie]

Thanks @hoyosjs !

Unless someone thinks differently on this, I'm inclined to close this issue as won't fix for now. Thoughts?

[dougbu]

WFM @markwilkie

we can always reopen if the arithmetic changes or we get new information