Have dotnet-sdk-3.0 Linux packages depend on exact dotnet-targeting-pack version

[dagood]

Details here are for RPM packages, but we are in the same situation with Debian packages:

$ repoquery --requires dotnet-sdk-3.0 | grep dotnet-targeting
dotnet-targeting-pack-3.0

This doesn't express the actual dependency. dotnet-sdk-3.0 contains a file specifying that it needs a specific full major.minor.patch of the targeting pack. This means that e.g. dotnet-sdk-3.0-3.0.101 is compatible with dotnet-targeting-pack-3.0-3.0.0, but patching to dotnet-targeting-pack-3.0-3.0.1 breaks the SDK.

See dotnet/core#3868 for @craigjbass's report on the way this can break SDK scenarios.

Right now we're relying on the latest version of dotnet-sdk-3.0 and dotnet-targeting-pack-3.0 on our Linux repos to happen to work with each other. This may also cause people headaches if they try to set up an old SDK version, because RPM doesn't know how to find the right targeting pack for them.

Mitigations:

• The latest versions should match up and work if our releases go smoothly. This loose dependency is causing additional pain when the repo is messed up.
• The SDK will download targeting packs if they don't exist. This only works for online build scenarios.

/cc @leecow @omajid @dleeapho

[nguerrera]

This seems like a good safeguard.

[omajid]

I am a little confused about what this means in a source-build context

This means that e.g. dotnet-sdk-3.0-3.0.101 is compatible with dotnet-targeting-pack-3.0-3.0.0, but patching to dotnet-targeting-pack-3.0-3.0.1 breaks the SDK.

Is that because dotnet-sdk-3.0-3.0.101 and dotnet-targeting-pack-3.0-3.0.0 are coherent? Or is dotnet-sdk-3.0 using a build produced by an older SDK build? Is dotnet-targeting-pack-3.0-3.0.1 from a yet different set of sources?

If we have source-build building .NET Core sources tagged as 3.0.5, would (should?) that be producing the right combo of, say, dotnet-sdk-3.0-3.0.105 and dotnet-targeting-pack-3.0-3.0.0?

The SDK will download targeting packs if they don't exist. This only works for online build scenarios.

Is there some way to verify a set of RPM packages (or a .NET Core SDK installation) has a self-consistent set of sdk-and-targetting-packs even if the machine is online?

[omajid]

cc @tmds @RheaAyase

[dagood]

To clarify what the Microsoft build is doing, yes, the new SDK depends on the previously built targeting pack bits. Here are the GA and 3.0.1 servicing build outputs:

• 3.0.0 product: 3.0.100 SDK and 3.0.0 netcoreapp targeting pack
• 3.0.1 product: 3.0.101 SDK and 3.0.1 netcoreapp targeting pack
  • This targeting pack should not have been released, because 3.0.101 still strongly depends on 3.0.0 from the previous build.
• In 3.0.2+ we won't create a targeting pack at all unless it carries a patch (so it can't be accidentally used or published) [release/3.0] Add servicing config: project skip infrastructure core-setup#8827

I thought about how source-build works with this in https://github.com/dotnet/core-setup/issues/8735, because yeah, it's not friendly. What we have now is that Core-Setup will produce the targeting pack, branded as the last-released patch version. I have a small table of Microsoft builds vs. source-build in dotnet/arcade#4318 to demonstrate.

Note the big assumption here: that it's possible to take e.g. v3.0.5 sources and build a targeting pack with the same contents as v3.0.0, then brand it as 3.0.0. There's risk of unwanted changes getting in v3.0.5 that won't be detected by the Microsoft build because it uses old binaries, but change how the source-build product functions.

This is a general problem that I have dotnet/source-build#923 filed for, which applies to more than just targeting packs. Most notably, the CoreFX OOB packages, which aren't in the netcoreapp shared framework, but could be used by SDK tooling repos and/or the ASP.NET Core shared framework.

Is there some way to verify a set of RPM packages (or a .NET Core SDK installation) has a self-consistent set of sdk-and-targetting-packs even if the machine is online?

What I did is notice that after I dotnet new'd a new project after a fresh install, I suddenly had ~/.nuget/packages/microsoft.netcore.app.ref/3.0.0. 😄 But a more static way post-install would be something like this:

$ for x in /usr/share/dotnet/packs/*; do
    echo $x
    ls $x
    grep -A 1 $(basename $x) /usr/share/dotnet/sdk/3.0.101/Microsoft.NETCoreSdk.BundledVersions.props
    echo
  done

/usr/share/dotnet/packs/Microsoft.AspNetCore.App.Ref
3.0.1
                              TargetingPackName="Microsoft.AspNetCore.App.Ref"
                              TargetingPackVersion="3.0.1"

/usr/share/dotnet/packs/Microsoft.NETCore.App.Host.linux-x64
3.0.1
# (Note, this one doesn't work because of the RID, searching for Microsoft.NETCore.App.Host gets the expected version.)

/usr/share/dotnet/packs/Microsoft.NETCore.App.Ref
3.0.1
                              TargetingPackName="Microsoft.NETCore.App.Ref"
                              TargetingPackVersion="3.0.0"

/usr/share/dotnet/packs/NETStandard.Library.Ref
2.1.0
                              TargetingPackName="NETStandard.Library.Ref"
                              TargetingPackVersion="2.1.0"

(This is with the bad 3.0.1 netcoreapp targeting pack installed, can see the mismatch.)

[tmds]

we won't create a targeting pack at all unless it carries a patch

Separate packages and versions are good for two things:

• The user doesn't have to upgrade all packages
• It's possible to rebuild only specific packages

imo, I don't consider these really important as a source-build user:
I don't mind if the user has to install more packages.
And I prefer building one while than understanding how it is composed of parts and what the dependencies are between the parts.

For source-build I think we should avoid the complexity of too many package versions and separately buildable parts.

With the current document (https://docs.microsoft.com/en-us/dotnet/core/build/distribution-packaging) we are using 3 version numbers: runtime version, aspnetcore runtime version and sdk version. This means we could build and distribute those separately.

The distribution packaging doc also says:

Dependencies between packages should use an equal or greater than version requirement. For example, dotnet-sdk-2.2:2.2.401 requires aspnetcore-runtime-2.2 >= 2.2.6. This makes it possible for the user to upgrade their installation via a root package (for example, dnf update dotnet-sdk-2.2).

This enables dependent packages to be updated separately. And updating packages from a root package.

[nguerrera]

It is a grave error to add/remove surface area in a patch. The system is built on the assumption that a given two part TFM adresses fixed surface area of the framework. The central promise of a reference assembly is that if you build against it you can run on that TFM regardless of its implementation that might or might not have patches on the target machine.(Previews break this rule, but that's separate.)

So if we do ship new targeting packs every patch we still have to make sure they don’t change somehow.

And then after being extra careful that we’re shipping the same thing, we would have to inflict extra download time and upgrade time to get the same files every month.

There is not only the extra install time but also consider that targeting packs are acquired from nuget when not installed, for example if you build for an older tfm than sdk’s runtime and haven’t the tp for it. So then every month you’d start redownloading these if you use this facility.

I think it’s helpful for reinforcing understanding that these don’t change to have them not patch redundantly.

[dagood]

I finished up a draft of a doc I started writing about this problem and put it up for PR at dotnet/source-build#1389. I tried to summarize the problems with trying to make source-build act more like the Microsoft build for these, what the existing solution has been, and the risk. I don't have any great ways forward here--the ideal (build everything at the correct commits) places burden on a lot of people and I don't see a middle ground that's safe wrt. the risk of artifacts changing that shouldn't have changed.

[tmds]

I'm trying to understand what is the root cause:

• is source-build building a targeting-pack and sdk which are not compatible?
• or does the sdk not understand it can use a newer targeting-pack?
• something else?

[dagood]

is source-build building a targeting-pack and sdk which are not compatible?

I haven't checked the source-built SDK (I really want dotnet/source-build#1345 so I can do this whenever), I'm not sure what it currently produces. "Compatible" is also hard to answer. If the targeting pack is bit-for-bit identical, then it's obviously compatible, but Microsoft builds aren't deterministic (I believe they can't be due to signing), and it's hard to evaluate once they're not identical.

This is the kind of risk we face by assuming current sources can build the old targeting packs, when that isn't how the Microsoft build works--not quite knowing how compatible things are. Figuring out ways to evaluate this (tests, static analysis) can help mitigate.

More about all that at dotnet/source-build#1389.

or does the sdk not understand it can use a newer targeting-pack?

This one is the topic of this thread: the SDK doesn't even find the newer-versioned pack directory. (It also wouldn't find a newer NuGet package, if it came to that, because version match is exact even there.) I've been thinking of it as part of compatibility. This problem is shared by the Microsoft build and source-build.

[nguerrera]

does the sdk not understand it can use a newer targeting-pack?

Let's take a step back here...

A targeting pack consists of reference assemblies, which consist entirely of surface area without implementation. By design and definition, the surface area of the frameworks are tied to TFM (target framework moniker) and we do not include the patch in the TFM. This allows code compiled for a given TFM to work irrespective of patch version of the ultimate runtime environment.

The targeting pack's version has three parts, like all nuget packages, but you should not interpret the third part as mapping to a specific framework patch number. For example, targeting pack Microsoft.NETCore.App.Ref 6.0.2 would be the second revision to the targeting pack for netcoreapp6.0. It has nothing do with the security fixes in version 6.0.2 of the runtime.

For the most part, revisions to targeting packs should be limited. There are very few things you can revise. (We did revise the ASP.NET Core 3.0 targeting pack to add missing type forwards that should have been there all along. I do not expect this to be the common case at all.)

For any given TFM and shared framework, the SDK is bound to a 3 part (or 3 part with prerelease) specific version of the targeting pack. There are practical reasons for this, such as determinism and performance when pulling targeting pack from nuget. This means that it will not just use 3.0.1 because there is one, it has to be taught to start using 3.0.1. I do not recommend that source build change the TP version and this mapping. It is possible to build 3.0.0 from source either by building at the same commit as the official 3.0.0 pack on nuget.org, or building a newer commit that nevertheless produces 3.0.0, with validation that the content hasn't changed.

With respect to the mechanics of this, my understanding is that reference assemblies need to be built at the very beginning of the framework build so that they can be consumed by the rest of the build. Furthermore, I heard the idea floating around of pipelining the larger product build to start building downstream repos as soon as reference assemblies are available and defer testing until all of the implementations are available. To give better build throughput. If this were formalized, it seems like it shouldn't be especially hard to allow for the reference build phase to be performed at a different commit.

[marcpopMSFT]

Old issue triage: We believe a few releases ago, we started updating the targeting packs with our various packages so I think this is closed.