Build targeting packs in the repos, not source-build-reference-packages

[dagood]

Core-Setup (Runtime) and AspNetCore produce targeting packs. The targeting pack builds are not always enabled during the servicing lifecycle, so we decompile them to build from source in source-build-reference-packages. This causes continuous work during servicing: each release, we have to discover whether new targeting packs are necessary, and decompile them if so.

We also need to be careful not to build targeting packs from source (dotnet/runtime#32982), since they may have incomplete platform manifests: dotnet/runtime#1622.

Instead, repos should be able to build targeting packs, and source-build should build them that way. The issue is that core-setup at e.g. 3.1.3 might not be able to produce a 3.1.0 targeting pack due to source changes since Microsoft shipped its 3.1.0 release. A way to resolve that is being proposed by #1389.

[dagood]

NETStandard.Library.Ref is also a targeting pack, with the same servicing policy as the others. It's less of a known risk because it doesn't have a platform manifest.

I'm posting this as a note to check up on that targeting pack when moving the targeting packs to build in the repos. In theory, it might be possible to build NETStandard.Library.Ref in the source repo before we change the other packs. (As of writing, we actually were missing the netstandard 2.1.0 targeting pack from SBRP--that inspired this note.)

[ericstj]

We want to have serviceable ref-packs. Why wouldn't we always build ref-packs during source build and just selectively version them? Does @jkoritzinsky's changes in the shared framework solve the platform manifest problem? I believe that's more of an issue for runtime, ASP.NET doesn't differ by RID IIRC.

[dagood]

IIRC it does fix the platform manifest part, but there's still the issue of whether e.g. v7.0.13 can really produce an equivalent pack to v7.0.0. Are there tests are in place that verify nobody's unintentionally added something that ends up in the targeting pack?

Where I'm thinking this would cause problems is when a source-built targeting pack included in the source-built 7.0.13 SDK doesn't have the same surface area as the Microsoft-built 7.0.0 targeting pack that's redistributed in the 7.0.13 Microsoft-built SDK. (Compile-time incompatibility for SDK users.)

I wrote out some more about this a while back at https://github.com/dotnet/source-build/tree/master/Documentation/planning/nongranular-servicing-readiness, although it focuses on the platform extension packages. With the extension packages, we currently assume it'll work if we force them to build. (I believe this has bit us several times recently.) For the targeting packs, building them from constant source code in SBRP is a way to know for sure they definitely won't change.

@dseefeld @crummel @MichaelSimons

[dagood]

We want to have serviceable ref-packs.

This is separate--new versions of the runtime and aspnetcore targeting packs are added to SBRP whenever a new version is produced. (This has happened a decent number of times for aspnetcore.) It would be great for source-build to no longer have to do that, but serviceability is not blocked.

[ericstj]

My point of bringing up "serviceable" ref-packs here was that we previously couldn't safely rebuild the ref-pack as it might contain inadvertent changes, like new file versions in the platform manifest. It turns out serviceable ref-packs and source build have similar requirements here. So long as we're maintaining servicability of our ref-packs it should be safe to enable them for source-build.

Are there tests are in place that verify nobody's unintentionally added something that ends up in the targeting pack?

We closely scrutinize all servicing changes, so that shouldn't be a concern. I believe we've suggested enabling ApiCompat between GA (as implementation) and Serviced (as contract) as a solution but folks have deemed it unnecessary. It'd be a largely trivial addition similar to https://github.com/dotnet/runtime/blob/652dcc06ea1a8390fa69633f677dc1015a71554b/src/libraries/shims/ApiCompat.proj#L107-L121.

[dagood]

Ah, I thought serviceable just meant "we can build a new version in servicing", but I guess not. 😄 I don't know what it does mean, but that sounds more reasonable now.

Sounds reasonable to go ahead with this.

[ericstj]

Sorry about that, yeah when I say "servicable" I mean everything we need to do to make sure it's safe to easily build and ship a new version in servicing. FYI @Anipik

[MichaelSimons]

[Triage] Targeting packs for runtime and aspnetcore were added to source-build in 6.0. This was required with the addition of analyzers in the targeting packs.