glibc: 2.27 -> 2.30 WIP

[lblasc]

Motivation for this change

Bump to latest glibc.

Started to fix core packages, managed to build and test rustc on x86_64.

• can someone please point me to latest bootstrap files for i686 and x86_64 on tarballs.nixos.org so I can remove my private mirror from pkgs/stdenv/linux/bootstrap-files/* ? @grahamc @Mic92
• hydra jobset for this would be great
• test darwin

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

Notify maintainers

cc @edolstra

[jonringer]

You're a madman, and i support you on your journey ❤️

However, such a universal package may cause a lot more heart-ache than it's worth. (don't want you to get burnt out)

[Mic92]

cc @LnL7

[vcunat]

Thanks for the fast reaction. I guess I have to ping @edolstra, as I don't know about anyone else.

[vcunat]

@Ma27

I talked with a few people at FOSDEM how we could improve the situation with bootstrap files (e.g. by using stuff from a binary cache by default), but this is something we should take care of after the branchoff.

There seems to be some misunderstanding. Bootstrap files are substituted from cache.nixos.org (if present); see e.g. log from busybox a couple posts above.

• The regular Hydra job as of now doesn't produce these as separate files, so it's the same files but nested deeper in other /nix/store paths and can't be directly re-used. Still, it's not difficult to nix build $file and then nix-prefetch-url file://$file
• Users with different location of nix store can't use the official binary cache (the hash parts are changed), so it's good to have working fixed-output fetches for them to bootstrap from. Hence tarballs.nixos.org
• These fetches also serve as backup in case the files got garbage-collected from the cache, though I suspect in our case that's just a theoretical possibility for now.

Maybe I forgot some point. We certainly had at least one GitHub discussion about the (non-)utility of tarballs.nixos.org.

[edolstra]

@vcunat Thanks, I've copied the files.

[vcunat]

Confirmed, except for the optional one (but that one doesn't matter).

[edolstra]

No, I copied it as well. Note that you have a typo in the URL (http://tarballs.nixos/org...).

[lblasc]

@vcunat @Ma27 it was a pleasure working with you guys! I've started this as a learning experience to investigate how NixOS bootstrapping works, I'm very pleased to have contribution which rebuilds the whole dependency tree 🔥 on three major archs 😁

[vcunat]

Oh, right, typo; pushed as cf24265. First x86_64 binaries should start compiling today; the rest when staging-next get rotated.

@lblasc: it's a bit unlucky that it was a difficult case and took us so much time – about half a year. Usually changes aren't so cumbersome, even mass rebuilds get merged basically every week.

[vcunat]

Repeated note: 32-bit ARMs are quite certainly broken by this PR. /cc @dezgeg (again) Regenerating bootstrap tools for them might suffice to fix them, but I don't have a good way of testing the builds natively.

I suspect cross-building these platforms might keep working, except I now see that this PR broke evaluation of (all?) cross stuff 🤕

[Ma27]

I suspect cross-building these platforms might keep working, except I now see that this PR broke evaluation of (all?) cross stuff

Can you please point me to a Hydra eval where I can see the errors?

[jcumming]

Uh-oh. These bootstrap tools no work on SLES 11-SP4 3.0.101-108.87-default kernels:

Unpacking the bootstrap tools...
Patching the bootstrap tools...
FATAL: kernel too old

[Ma27]

Just to properly understand your issue: you're using Nix on Sles11sp4 and our new bootstrap tools break the setup, right?

However I'd like to point out that I'm pretty much against putting additional effort in fixing stuff for unmaintained versions of other distros (well unless you're paying for LTSS, but that's another story), but that's just my opinion :)

[vcunat]

@jcumming: did it even work before this PR? I've never looked into this for SLES/SLED. We haven't changed this part for quite a long time:

"--enable-kernel=3.2.0" # can't get below with glibc >= 2.26

We have special patch to allow RHEL 6 -like kernels. That reminds me I last verified the real compatibility with that on glibc-2.26-131... so even though you don't get this nice refusal, some parts might not work on those kernels anymore. Well, today I suspect we might even drop RHEL 6 support 🤷‍♀️ I personally don't feel it's important anymore.

[vcunat]

To be clear, it isn't that much about bootstap... but about running anything built with our libc (i.e. with our stdenv).

[vcunat]

Actually, I think previous service packs with 2.6.32 kernel would not give you the error message, by co-incidence with RHEL 6 :-)