Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use live apphost when publishing ilc as singlefile #105004

Open
wants to merge 28 commits into
base: main
Choose a base branch
from
+33 −21
Open

Use live apphost when publishing ilc as singlefile #105004

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
8e244d6
Use live apphost when publishing ilc as singlefile
am11 Jul 17, 2024
93c6d88
Block ILCompiler.csproj build on mobile
am11 Jul 17, 2024
b00efad
Invert condition
am11 Jul 17, 2024
351856c
Limit the scope of build order
am11 Aug 29, 2024
46b01fd
Merge branch 'main' into feature/freebsd-port/outputrid
am11 Sep 2, 2024
b90b7fd
Merge branch 'main' into feature/freebsd-port/outputrid
am11 Sep 12, 2024
97a0686
Address CR feedback
am11 Sep 13, 2024
5d7dba3
Merge dotnet/main into feature/freebsd-port/outputrid
am11 Sep 13, 2024
d46a6fd
Update src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj
am11 Sep 13, 2024
713e9ec
Update RID
am11 Sep 13, 2024
8be0a6a
Use first step in CI (for now)
am11 Sep 13, 2024
4684361
Add postBuildSteps for step 2
am11 Sep 13, 2024
e5471c8
Merge branch 'main' into feature/freebsd-port/outputrid
am11 Sep 13, 2024
5903321
fix
am11 Sep 13, 2024
1b81367
Revert to old main
am11 Dec 18, 2024
ea14512
Merge dotnet/main into feature/freebsd-port/outputrid
am11 Dec 18, 2024
2da74d3
Add multi-stages for community supported platforms
am11 Dec 18, 2024
e096e4a
Apply suggestions from code review
am11 Dec 19, 2024
1b9948c
Update toolAot.targets
am11 Dec 19, 2024
8d81ab4
Revert experiment
am11 Dec 19, 2024
a59b66a
Merge branch 'main' into feature/freebsd-port/outputrid
am11 Jan 8, 2025
fc810a3
Replace hardcoded platform arg
am11 Jan 9, 2025
e2344a9
Try with UseNativeAotForComponents=true
am11 Jan 9, 2025
d376752
Apply condition on versino directly
am11 Jan 9, 2025
ba71b35
Remove over-defensive condition
am11 Jan 9, 2025
4b0df9f
Revert UseNativeAotForComponents change
am11 Jan 9, 2025
bd6549a
Merge branch 'main' into feature/freebsd-port/outputrid
am11 Jan 9, 2025
011280d
Enable NativeAot runtime build on riscv64
am11 Jan 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions eng/Subsets.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,11 +121,11 @@

<PropertyGroup>
<!-- CLR NativeAot only builds in a subset of the matrix -->
<_IsCommunityCrossArchitecture Condition="'$(CrossBuild)' == 'true' and ('$(TargetArchitecture)' == 'loongarch64' or '$(TargetArchitecture)' == 'riscv64')">true</_IsCommunityCrossArchitecture>
<_NativeAotSupportedOS Condition="'$(TargetOS)' == 'windows' or '$(TargetOS)' == 'linux' or '$(TargetOS)' == 'osx' or '$(TargetOS)' == 'maccatalyst' or '$(TargetOS)' == 'iossimulator' or '$(TargetOS)' == 'ios' or '$(TargetOS)' == 'tvossimulator' or '$(TargetOS)' == 'tvos' or '$(TargetOS)' == 'freebsd'">true</_NativeAotSupportedOS>
<_NativeAotSupportedArch Condition="'$(TargetArchitecture)' == 'x64' or '$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'arm' or '$(TargetArchitecture)' == 'loongarch64' or ('$(TargetOS)' == 'windows' and '$(TargetArchitecture)' == 'x86')">true</_NativeAotSupportedArch>
<NativeAotSupported Condition="'$(_NativeAotSupportedOS)' == 'true' and '$(_NativeAotSupportedArch)' == 'true' and '$(_IsCommunityCrossArchitecture)' != 'true'">true</NativeAotSupported>
<UseNativeAotForComponents Condition="'$(NativeAotSupported)' == 'true' and '$(TargetOS)' == '$(HostOS)' and '$(TargetsLinuxBionic)' != 'true'">true</UseNativeAotForComponents>
<_NativeAotSupportedArch Condition="'$(TargetArchitecture)' == 'x64' or '$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'arm' or '$(TargetArchitecture)' == 'loongarch64' or '$(TargetArchitecture)' == 'riscv64' or ('$(TargetOS)' == 'windows' and '$(TargetArchitecture)' == 'x86')">true</_NativeAotSupportedArch>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should invert this condition; blacklist instead of whitelist the unsupported platforms.

<NativeAotSupported Condition="'$(_NativeAotSupportedOS)' == 'true' and '$(_NativeAotSupportedArch)' == 'true'">true</NativeAotSupported>
<_IsCommunityCrossArchitecture Condition="'$(CrossBuild)' == 'true' and ('$(TargetArchitecture)' == 'loongarch64' or '$(TargetArchitecture)' == 'riscv64')">true</_IsCommunityCrossArchitecture>
<UseNativeAotForComponents Condition="'$(NativeAotSupported)' == 'true' and '$(TargetOS)' == '$(HostOS)' and '$(TargetsLinuxBionic)' != 'true' and '$(_IsCommunityCrossArchitecture)' != 'true'">true</UseNativeAotForComponents>

<!-- If we're building clr.nativeaotlibs and not building the CLR runtime, compile libraries against NativeAOT CoreLib -->
<UseNativeAotCoreLib Condition="'$(TestNativeAot)' == 'true' or ($(_subset.Contains('+clr.nativeaotlibs+')) and !$(_subset.Contains('+clr.native+')) and !$(_subset.Contains('+clr.runtime+')) and !$(_subset.Contains('+clr.corelib+')))">true</UseNativeAotCoreLib>
Expand Down Expand Up @@ -360,9 +360,9 @@
$(CoreClrProjectRoot)tools\PdbChecker\PdbChecker.csproj;
$(CoreClrProjectRoot)tools\AssemblyChecker\AssemblyChecker.csproj" Category="clr" Condition="'$(DotNetBuildSourceOnly)' != 'true'"/>
<!-- skip the architectures that don't have LKG runtime packs -->
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\aot\crossgen2\crossgen2_publish.csproj" Condition="'$(NativeAotSupported)' == 'true'" Category="clr" />
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\aot\crossgen2\crossgen2_publish.csproj" Condition="'$(NativeAotSupported)' == 'true' and '$(StageOneBuild)' != 'true'" Category="clr" />
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jkoritzinsky ILCompiler_publish.csproj is not listed in this file (or anywhere else). Was it an oversight or something for the future?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there was an intent to hook it up for the Microsoft.DotNet.ILCompiler package, but I think it didn't quite happen. Looks like we still publish from the ILCompiler.csproj project and ship that. I think the build needs to be adjusted to hook this up correctly (use ILCompiler_publish.csproj to publish ilc and remove the publishing logic from ILCompiler.csproj)

<ProjectToBuild Include="$(CoreClrProjectRoot)tools\aot\ILCompiler.Build.Tasks\ILCompiler.Build.Tasks.csproj" Category="clr" Condition="'$(NativeAotSupported)' == 'true'" />
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\aot\ILCompiler\ILCompiler.csproj" Category="clr" Condition="'$(NativeAotSupported)' == 'true'" />
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\aot\ILCompiler\ILCompiler.csproj" Category="clr" Condition="'$(NativeAotSupported)' == 'true' and '$(StageOneBuild)' != 'true'" />
<ProjectToBuild Include="$(CoreClrProjectRoot)nativeaot\BuildIntegration\BuildIntegration.proj" Category="clr" Condition="'$(NativeAotSupported)' == 'true'" />

<ProjectToBuild Condition="'$(NativeAotSupported)' == 'true' and ('$(CrossBuild)' == 'true' or '$(BuildArchitecture)' != '$(TargetArchitecture)' or '$(HostOS)' != '$(TargetOS)' or '$(EnableNativeSanitizers)' != '')" Include="$(CoreClrProjectRoot)tools\aot\ILCompiler\ILCompiler_inbuild.csproj" Category="clr" />
Expand Down
20 changes: 19 additions & 1 deletion eng/pipelines/runtime.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -493,8 +493,26 @@ extends:
jobParameters:
testScope: innerloop
nameSuffix: CoreCLR
buildArgs: -s clr+libs+host -c $(_BuildConfig) -rc Checked
buildArgs: -s clr+libs+host -c $(_BuildConfig) -rc Checked -p:StageOneBuild=true
timeoutInMinutes: 120
postBuildSteps:
- script: |
echo Running $(Build.SourcesDirectory)/build$(scriptExt) \
${{ variables.debugOnPrReleaseOnRolling }} \
-s clr.tools+packs -rc Checked -cross \
-os $(osGroup) \
-a $(archType) \
-c $(_BuildConfig) \
-p:StageTwoBuild=true

$(Build.SourcesDirectory)/build$(scriptExt) \
${{ variables.debugOnPrReleaseOnRolling }} \
-s clr.tools+packs -rc Checked -cross \
-os $(osGroup) \
-a $(archType) \
-c $(_BuildConfig) \
-p:StageTwoBuild=true
displayName: Build clr.tools and packs
condition: >-
or(
eq(stageDependencies.EvaluatePaths.evaluate_paths.outputs['SetPathVars_coreclr.containsChange'], true),
Expand Down
6 changes: 4 additions & 2 deletions eng/targetingpacks.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,8 +163,10 @@
Condition="'$(UseLocalTargetingRuntimePack)' == 'true'"
AfterTargets="ResolveTargetingPackAssets">
<ItemGroup>
<RuntimeFramework Version="$(ProductVersion)"
Condition="'%(RuntimeFramework.FrameworkName)' == '$(LocalFrameworkOverrideName)'" />
<RuntimeFramework Condition="'%(RuntimeFramework.FrameworkName)' == '$(LocalFrameworkOverrideName)'">
<Version Condition="'$(StageTwoBuild)' != 'true'">$(ProductVersion)</Version>
<Version Condition="'$(StageTwoBuild)' == 'true'">$(BundledNETCoreAppPackageVersion)</Version>
</RuntimeFramework>
</ItemGroup>
</Target>
</Project>
6 changes: 0 additions & 6 deletions eng/toolAot.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,6 @@

<PropertyGroup Condition="'$(AotOrSingleFile)' == 'true' and '$(UseNativeAotForComponents)' != 'true'">
<PublishSingleFile>true</PublishSingleFile>
<PublishReadyToRun>true</PublishReadyToRun>
<!-- Disable crossgen on NetBSD, illumos, Solaris, and Haiku for now. This can be revisited when we have full support. -->
<PublishReadyToRun Condition="'$(TargetOS)' == 'netbsd' Or '$(TargetOS)' == 'illumos' Or '$(TargetOS)' == 'solaris' Or '$(TargetOS)' == 'haiku'">false</PublishReadyToRun>
<!-- Disable crossgen on FreeBSD when cross building from Linux. -->
<PublishReadyToRun Condition="'$(TargetOS)' == 'freebsd' and '$(CrossBuild)' == 'true'">false</PublishReadyToRun>
<PublishReadyToRunComposite>$(PublishReadyToRun)</PublishReadyToRunComposite>
</PropertyGroup>

<!-- Allow overriding the LKG ILCompiler package with a specific version. -->
Expand Down
6 changes: 0 additions & 6 deletions src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,8 @@

<Import Project="ILCompiler.props" />


<!-- BEGIN: Workaround for https://github.com/dotnet/runtime/issues/67742 -->
<PropertyGroup Condition="'$(BuildingInsideVisualStudio)' != 'true'">
<PublishDir>$(RuntimeBinDir)ilc-published/</PublishDir>
<PublishAot Condition="'$(UseNativeAotForComponents)' == 'true'">true</PublishAot>
<PublishReadyToRun Condition="'$(UseNativeAotForComponents)' != 'true'">true</PublishReadyToRun>
<PublishSingleFile Condition="'$(UseNativeAotForComponents)' != 'true'">true</PublishSingleFile>
<PublishTrimmed Condition="'$(UseNativeAotForComponents)' != 'true'">true</PublishTrimmed>
</PropertyGroup>

<Target Name="PublishCompiler"
am11 marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
2 changes: 2 additions & 0 deletions src/coreclr/tools/aot/ILCompiler/ILCompiler.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,11 @@
<GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
<EnableDefaultEmbeddedResourceItems>false</EnableDefaultEmbeddedResourceItems>
<Configurations>Debug;Release;Checked</Configurations>
<UseLocalTargetingRuntimePack Condition="'$(StageTwoBuild)' == 'true'">true</UseLocalTargetingRuntimePack>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a problem for incremental builds? Should the build with LKG target and the build local targeting pack go into separate directories?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This project is not built in stage 1, so incremental build does not come into play.

</PropertyGroup>

<Import Project="../AotCompilerCommon.props" />
<Import Project="$(RepositoryEngineeringDir)targetingpacks.targets" Condition="'$(StageTwoBuild)' == 'true'" />

<PropertyGroup>
<SelfContained>true</SelfContained>
Expand Down
2 changes: 2 additions & 0 deletions src/coreclr/tools/aot/crossgen2/crossgen2.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,11 @@
<EnableDefaultEmbeddedResourceItems>false</EnableDefaultEmbeddedResourceItems>
<Configurations>Debug;Release;Checked</Configurations>
<RunAnalyzers>false</RunAnalyzers>
<UseLocalTargetingRuntimePack Condition="'$(StageTwoBuild)' == 'true'">true</UseLocalTargetingRuntimePack>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be in toolAot.targets so that it applies to all tools published with Aot automatically?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can try :)

Suggested change
<UseLocalTargetingRuntimePack Condition="'$(StageTwoBuild)' == 'true'">true</UseLocalTargetingRuntimePack>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, it is giving errors like /__w/1/s/src/libraries/Directory.Build.targets(120,3): error MSB4011: "/__w/1/s/eng/targetingpacks.targets" cannot be imported again. It was already imported at "/__w/1/s/eng/toolAot.targets (7,3)". This is most likely a build authoring error. This subsequent import will be ignored. [/__w/1/s/src/libraries/System.Runtime.InteropServices/gen/Microsoft.Interop.SourceGeneration/Microsoft.Interop.SourceGeneration.csproj] [/__w/1/s/Build.proj]

We will need to add tracking. targetingpacks.targets is used in 29 places. I'm leaving it as is for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A future change should import targetingpacks.targets in the repo root un-conditionally and enable the features in it via feature knobs. I should have done that in the first place back when I introduced the file.

</PropertyGroup>

<Import Project="../AotCompilerCommon.props" />
<Import Project="$(RepositoryEngineeringDir)targetingpacks.targets" Condition="'$(StageTwoBuild)' == 'true'" />

<ItemGroup Label="Embedded Resources">
<EmbeddedResource Include="Properties\Resources.resx">
Expand Down
Loading