Get Julia building cleanly on Alpine Linux

[ararslan]

I received an email from someone who is particularly interested in using Julia on Alpine Linux, which uses musl libc rather than glibc. They gave me a summary of what existing changes could be included, ideally in 1.2, to get Julia building cleanly on Alpine. I've reproduced the email below.

I saw that you merged 2 patches needed for Julia compilation with musl in time for 1.2.0-rc2. There's now some final patches being merged for the final 1.2.0 release.
#32592

Julia still doesn't compile cleanly, but it's very close! 3 other patches are currently needed, as fredrikekre apply them in his Alpine repository.
https://github.com/fredrikekre/julia-alpine

The first patch is #31886
Really simple patch which shouldn't affect anyone, but allow compilation with musl.
There should be zero problems in merging for 1.2.0.

The second patch is more intrusive.
#31595

The last one is here.
https://github.com/fredrikekre/julia-alpine/blob/master/patches/revert-disable-BB-for-gmp-and-mpfr.diff

If we could add these patches in time for 1.2.0 then Julia would compile cleanly on Alpine Linux and the Julia organization could provide official musl binaries for users.

I'd be particularly interested in feedback from @fredrikekre, whose julia-alpine repository is referenced above, as well as the rest of the usual release suspects to determine an appropriate timeline and whether any of the above changes are sufficiently minimal to include in the next 1.2.0 release candidate.

[ararslan]

The first two patches here are pretty safe:

• Disable building of tests for libunwind #31886 disables building tests for libunwind. By the developers' own admission, these are unlikely to pass anyway for platforms other than x86-64 Linux with glibc, built with GCC.
• Add dSFMT and Objconv BB tarballs #31595 enables BinaryBuilder for dSFMT and Objconv.

The julia-alpine patch referenced toward the end is the problematic one; using BinaryBuilder for GMP and MPFR was reverted due to performance regressions. If we're to reenable those, it should be conditionally, though by design musl refuses to be easily detectable.

[ararslan]

The release folks have decided not to pursue this for 1.2. We'll target 1.3, by which time new GMP and MPFR binaries from BinaryBuilder should be available, making the last point above a non-issue.

[fredrikekre]

#31886 is not needed if we make a new BB release for libunwind that includes JuliaPackaging/Yggdrasil#13

[MasonProtter]

Is this still planned for 1.3 or now being pushed back to 1.4 since we're already at the RC stage for 1.3?

[KristofferC]

1.4. We don't, for example, have functioning GMP + MPFR binaries yet.

[StefanKarpinski]

Well, we have functioning binaries, they're just much slower so we don't use them.

[ecsx1]

#34155 was merged and Alex and I can't reproduce the segfault in #28805 with the newest release.
Any update on the status here? I could help testing whatever is needed with a native bare-metal Alpine machine.

[ecsx1]

1.4

I see that Julia v1.4.0-rc2 is now available but there are no musl binaries.
Is the new milestone 1.5 or is there a chance the final 1.4 release will have binaries for musl?

[fredrikekre]

https://github.com/fredrikekre/julia-alpine/releases/tag/v1.4.0-rc2%2B0

[ViralBShah]

I thought we've never released official julia musl binaries in the past.

[StefanKarpinski]

If we got Julia building in binary builder we’d get all the things for free. Just saying.

[ararslan]

I thought we've never released official julia musl binaries in the past.

Nope, this would be a newly supported platform. We've had partial support for quite a while but it always required some weird workarounds and had issues.

[ecsx1]

Julia now builds from source without any patches on Alpine.
It's as simple as

git clone https://github.com/julialang/julia
cd julia
make

After a few minutes a beautiful binary is born.

julia/usr/bin/julia: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped

Base Julia and all seems to be working great.
It's 100% usable after that as long as you don't need any packages.

Expect issues if you try to packages which depend on non-Julia code.

julia> using Plots

ERROR: LoadError: LoadError: InitError: could not load library "/home/ecs/.julia/artifacts/7ff9517379466c2ff17c2bc63f18cd90f943f387/lib/libfreetype.so"

julia> using SymbolicUtils

ERROR: LoadError: LoadError: InitError: could not load library "/home/ecs/.julia/artifacts/d822684e5611ae99067a1b9bc38e0896bf9a1eac/lib/libgomp.so.1"

julia> Pkg.add("Cxx")

┌ Error: Error building `Cxx`:
│ ERROR: LoadError: could not load library "libLLVM-9.0.1"

And so on... Basically any packages that have external deps.

But as this issue is specific for building core julia from source I thought I'd let you know we reached 100% on that!
Julia builds cleanly now so this issue is resolved.

The other issues are related to packaging and should be solved by JLL, Yggdrasil and friends.
I know Elliot and others are working hard on that and I'm here to help if I can. You can send me any requests to test stuff on a bare metal Alpine box.

Congrats on the milestone.

[ViralBShah]

Thanks. I suggest closing this issue and opening specific issues related to binaries. Those issues may being possibly even to other orgs.

[ViralBShah]

We do build musl-linked binaries in yggsrasil but some packages are not successful.

[ararslan]

Note that a few tests still fail on Alpine, but that's great that things are otherwise working well!

[ViralBShah]

Note that a few tests still fail on Alpine, but that's great that things are otherwise working well!

We should open individual issues for those - so that we get them fixed over time!