Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix building on ARMhf userland on top of AARCH64 kernel #97929

Merged
merged 2 commits into from
Feb 6, 2024

Conversation

filipnavara
Copy link
Member

@filipnavara filipnavara commented Feb 3, 2024

Raspberry Pi 4/5 supports running 32-bit user land with a 64-bit Linux kernel. Detect that and install linux-arm version of .NET instead of linux-arm64. The arm64 version doesn't work in that scenario.

Should I submit the relevant bits as dotnet/arcade PR?

Related: dotnet/install-scripts#435

@ghost ghost added the community-contribution Indicates that the PR has been added by a community member label Feb 3, 2024
@ghost
Copy link

ghost commented Feb 3, 2024

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

Raspberry Pi 4/5 supports running 32-bit Linux with a 64-bit kernel. Detect that and install linux-arm version of .NET instead of linux-arm64. The arm64 version doesn't work in that scenario.

Should I submit the relevant bits as dotnet/arcade PR?

Author: filipnavara
Assignees: -
Labels:

area-Infrastructure-libraries, community-contribution

Milestone: -

@ghost
Copy link

ghost commented Feb 3, 2024

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

Raspberry Pi 4/5 supports running 32-bit Linux with a 64-bit kernel. Detect that and install linux-arm version of .NET instead of linux-arm64. The arm64 version doesn't work in that scenario.

Should I submit the relevant bits as dotnet/arcade PR?

Related: dotnet/install-scripts#435

Author: filipnavara
Assignees: -
Labels:

area-Infrastructure, community-contribution

Milestone: -

@am11
Copy link
Member

am11 commented Feb 4, 2024

Should I submit the relevant bits as dotnet/arcade PR?

Yes, anything that goes in eng/common needs to be added there so the next arcade sync PR doesn't overwrite the changes.

Fix building on ARMhf userland on top of AARCH64 kernel

Is it so that we supported this configuration at some point which is now broken? AFAIK, we never supported things like 32-bit unix process on 64 bit machine (-mx32 flag __ILP32__ macro etc.) on xarch either. Not sure if it's worth the trouble, but if that's all it takes, why not! 😅

@filipnavara
Copy link
Member Author

filipnavara commented Feb 4, 2024

Yes, anything that goes in eng/common needs to be added there so the next arcade sync PR doesn't overwrite the changes.

I'll submit dotnet/arcade PR when I get back to my machine on Monday.

Fix building on ARMhf userland on top of AARCH64 kernel

Is it so that we supported this configuration at some point which is now broken? AFAIK, we never supported things like 32-bit unix process on 64 bit machine (-mx32 flag __ILP32__ macro etc.) on xarch either. Not sure if it's worth the trouble, but if that's all it takes, why not! 😅

There were definitely some attempts to support it (dotnet/arcade#13392).

I think this configuration is slightly more common than the x86/x64 mix. Raspberry Pi installer still offers it as one of the options in the official installer. Moreover, it happens to automatically choose the kernel based on the hardware, so if you swap an SD card from older Raspberry Pi into a new one you may end up with 64-bit kernel. Raspberry Pi 5 started ignoring the arm_64bit=0 flag in the config.txt file completely, so this mix is the only supported 32-bit configuration there. I realize that in practice you want to run full AArch64 Linux on it but CoreCLR currently doesn't support cross-building from ARM64 hosts.

@am11
Copy link
Member

am11 commented Feb 4, 2024

I think this configuration is slightly more common than the x86/x64 mix.

There are few differences:

  • 32-bit emulating on 64-bit (qemu)
  • x32 ABI (a 32-bit process compiled for 64-bit OS): we previously explored this scenario a bit (during Environment.ProcessArchitecture overhaul), but it was considered uninteresting configuration.
    • aarch64-linux-gnu_ilp32 is analogous to x86_64-linux-gnux32.
  • x86 OS running on x64 machine: ProcessArchitecture vs. ProcessorArchitecture APIs would need to be revisited at least?

I realize that in practice you want to run full AArch64 Linux on it but CoreCLR currently doesn't support cross-building from ARM64 hosts.

Yup, if all it takes is adding missing pieces in shell/cmake configurations, sounds like we should go for it. My hunch was supporting these configurations would require updating coreclr-pal as well (among other things), so this maybe the tip of iceberg. I don't mean to demotivate, just a heads-up; there may be possible dragons. :)

@filipnavara
Copy link
Member Author

My hunch was supporting these configurations would require updating coreclr-pal as well (among other things), so this maybe the tip of iceberg.

This is very different from the x32 or emulation scenarios. The whole user land here is standard ARMhf compiled binaries, instructions, ABI and everything, only the kernel is different. You cannot even run aarch64 binaries because there are no user land libraries to support that. The only problem is detection using uname -m and commands that report the kernel architecture.

I have been running this configuration for a week with both CoreCLR and NativeAOT builds, incl. test runs. I don’t expect (many) dragons ahead. In fact, the current off-the-shelf behavior on this system is that the build downloads linux-arm64 dotnet which cannot run, so it fails right away.

@am11
Copy link
Member

am11 commented Feb 4, 2024

This is very different from the x32 or emulation scenarios. The whole user land here is standard ARMhf compiled binaries, instructions, ABI and everything, only the kernel is different. You cannot even run aarch64 binaries because there are no user land libraries to support that. The only problem is detection using uname -m and commands that report the kernel architecture.

Yup, that's what I was trying to cover in third bullet. uname/struct utsname usages would need to be revisited.

I have been running this configuration for a week with both CoreCLR and NativeAOT builds, incl. test runs. I don’t expect (many) dragons ahead.

Nice! In that case, LGTM. :)
(btw, we don't have good 32-bit tests for Process{,or}Architecture on unix)

@@ -35,6 +35,10 @@ fi
case "$CPUName" in
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be nice to remove this script now that we have it in eng/common

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will check if we can remove the extra copy.

Similarly, any thoughts about the dotnet-install.sh copy? It’s not used at the moment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eng/common/dotnet-install.sh is used by eng/common/tools.sh.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not used by the top-level dotnet.sh which bit me quite a bit.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second look, the top-level dotnet.sh references eng/common/tools.sh which in turn downloads the official dotnet-install.sh from web.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's conditional on -a (if path exists) in tools.sh, so not unused. Maybe it is there for SB offline mode or some other repo use-case (like dotnet/arcade#14283), so leave it for now as out of PR scope? init-os-and-arch is definitely something we can dedup (as it was the plan dotnet/arcade#13947). It can also be cleaned up as a separate PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's conditional on -a (if path exists) in tools.sh, so not unused.

The $root path it checks is outside of eng/common normally though (e.g. in runtime we install the local dotnet into <repo root>/.dotnet) so the copy in common won't be used by tools.sh.

It is used by the Arcade msbuild targets to install dotnet: https://github.com/dotnet/arcade/blob/2fb543a45580400a559b5ae41c96a815ea14dac5/src/Microsoft.DotNet.Arcade.Sdk/tools/InstallDotNetCore.targets#L6-L11

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that begs a question... shouldn't it be used? (not going to resolve it in this PR, just asking)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea. From a quick look the eng/common/dotnet-install.sh is much simpler than the one from https://dot.net/v1/dotnet-install.sh, that was probably intentional?

@jkotas
Copy link
Member

jkotas commented Feb 5, 2024

Could you please resolve the conflict?

@filipnavara
Copy link
Member Author

Could you please resolve the conflict?

Yep, just working on it.

Copy link
Member

@jkotas jkotas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@jkotas jkotas merged commit d247d72 into dotnet:main Feb 6, 2024
177 of 180 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Mar 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-arm32 area-Infrastructure community-contribution Indicates that the PR has been added by a community member os-linux Linux OS (any supported distro)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants