Need Native AOT publishing test which validates packages aren't restored from feed

[mthalman]

There is currently a Native AOT test that gets run as part of the VMR scenario tests. But this test only validates that the publish worked. It doesn't verify a critical aspect of the source build requirements which are to avoid any dependency on a NuGet feed.

In fact the Native AOT publishing scenario test doesn't fulfill that requirement today. It ends up loading from a NuGet feed:

Installed runtime.linux-x64.Microsoft.DotNet.ILCompiler 10.0.0-alpha.1.25071.14 from https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet10/nuget/v3/index.json to /__w/1/s/src/scenario-tests/.packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/10.0.0-alpha.1.25071.14 with content hash pzLeGawFrvq3a3MyljNCHurehIhhVMEFS+K/xTgHxAM602oqVU/8qJIFtgxMIexNMN7aGB+QSXSraDBJjjueIg==.

This would be fixed by dotnet/scenario-tests#184. Having validation would have caught this. This kind of validation is already done for the self-contained publishing test at https://github.com/dotnet/sdk/blob/b324c2e074f7691705b85b6ed69be23d350164b8/src/SourceBuild/content/test/Microsoft.DotNet.SourceBuild.Tests/WebScenarioTests.cs#L45.

Note that such validation behavior should be defined in Microsoft.DotNet.SourceBuild.Tests since it is source build-specific; it should not be in the scenario tests repo.

cc @tmds

[tmds]

There is currently a Native AOT test that gets run as part of the VMR scenario tests. But this test only validates that the publish worked. It doesn't verify a critical aspect of the source build requirements which are to #1215.

I had assumed it wouldn't be able find an ilcompiler for linux-x64. I wasn't aware one was being provided through extra NuGet sources.

Note that such validation behavior should be defined in Microsoft.DotNet.SourceBuild.Tests since it is source build-specific; it should not be in the scenario tests repo.

I think it would be more clear to have the functional test for PublishAOT in a single test suite and have that test suite include the difference is between portable vs non-portable publish.

Placing the source-build specific validation in SourceBuild.Tests means duplicate code, overlapping coverage, and less understanding of how this works portable vs non portable.

cc @ViktorHofer @MichaelSimons

[mthalman]

Yeah, I agree with that. It should be straightforward to provide a property to scenario tests that indicates this state and have the tests do the validation accordingly.