State of dotnet/arcade's NuGet dependencies: how to use NuGet.Packaging in a new build task

[dagood]

• This issue is blocking
• This issue is causing unreasonable pain

Right now, it looks like a one-word summary of the NuGet dependencies in dotnet/arcade is "fragile"--but this is nothing new or surprising. This repo has two different versions of NuGet packages that it uses:

arcade/eng/Versions.props

Lines 40 to 41 in 1b5940f

	<NuGetVersioningVersion>4.4.0</NuGetVersioningVersion>
	<NuGetVersion>5.6.0-preview.2.6489</NuGetVersion>

The Arcade SDK uses only NuGetVersioningVersion:

arcade/src/Microsoft.DotNet.Arcade.Sdk/Microsoft.DotNet.Arcade.Sdk.csproj

Line 28 in 1b5940f

<PackageReference Include="NuGet.Versioning" Version="$(NuGetVersioningVersion)" />

NuGetVersion shows up in other SDK projects and MSBuild extension projects, and it's used for various NuGet package IDs.

There's also the long thread at #1965 that ended up with this workaround:

arcade/src/Microsoft.DotNet.Build.Tasks.Feed/Microsoft.DotNet.Build.Tasks.Feed.csproj

Lines 28 to 35 in 1b5940f

	<!-- Exclude NuGet package assemblies per https://github.com/dotnet/arcade/issues/1965
	This is a workaround for https://github.com/microsoft/msbuild/issues/5073. Required until #5073 is fixed or
	https://github.com/microsoft/msbuild/issues/5086 is available and we switch to that workaround -->
	<Target Name="ExcludeNuGetPackagesFromNetCore" BeforeTargets="GenerateNuspec">
	<ItemGroup>
	<_PackageFilesToExclude Include="%(_PackageFiles.Identity)" Condition="$([System.String]::Copy('%(_PackageFiles.FileName)').StartsWith('NuGet')) and $([System.String]::Copy('%(_PackageFiles.Directory)').Contains('$(_ExcludeNuGetAssembliesTargetFramework)'))" />
	</ItemGroup>
	</Target>

Q's

My first impression is that I shouldn't touch this at all. If I need to use NuGet.Packaging, instead of putting my task in the Arcade SDK, I should set up a new SDK or MSBuild extension project and use it there. Does this sound like a good precaution? For now I'm assuming it's what I have to do (and that it'll be sufficient to avoid problems...)

Is this something that should be revisited/simplified? I see dotnet/msbuild#5073 (bug) is still open, but dotnet/msbuild#5086 (workaround mentioned in the comment) is closed and presumably available. However... I don't know if this only addresses the workaround target, or also the underlying fragility and two-version usage.

/cc @chcosta @rainersigwald

[chcosta]

Today, in Arcade, the above workaround means that if you need to use NuGet.Packaging, you use the one that comes with the dotnet sdk instead of whatever your package specified. if you needed a newer version of NuGet.Packaging, you'd have to update the dotnet sdk that you're running on. Is that sufficient?

Last I heard, I thought there was an option to ease the version matching for assembly loading, but yeah, Rainer would probably know better and if we can / should revisit this.

[dagood]

Today, in Arcade, the above workaround means that if you need to use NuGet.Packaging, you use the one that comes with the dotnet sdk instead of whatever your package specified. if you needed a newer version of NuGet.Packaging, you'd have to update the dotnet sdk that you're running on. Is that sufficient?

Yep. (For source-build in particular, we actually directly compile against the SDK's DLLs to avoid having to deal with maintaining dependency versions that match what the SDK happens to include. But tightly binding with a specific SDK's DLLs is only reasonable because we don't distribute our task DLLs.)

A concern is that if I add a usage of a NuGet 4.4.0 API to the Arcade SDK, the .NET SDK might include 5.78.0-*, and the API is no longer binary compatible and breaks someone's build. I'm not sure how to verify this. It seems like the Arcade SDK is assuming the 4.4.0 API will always be forward-compatible, and the projects that compile against 5.6.0-* are closer to the .NET SDK, but still assuming compat.