Sane cross-compiling through bootstrapping

[Ericson2314]

Ok, this is it, my final and biggest top-level refactor for 2016!

I've factored things into separate PRs, so this builds on #21415 to provide a consistent idiom for bootstrapping which I extend and further utilize for cross compiling, and #21414 which cleans up the cross-compiling tests.

It should also eventually subsume #21388, but I am still getting things unbroken.

This isn't ready to merge (need to unbreak cross compiling completely) but is ready to review---only uninteresting bug fixes should be left.

---

OP

The first commit introduces a new abstraction for dealing with stdenv bootstrapping. It doesn't really make anything shorter, but at least it enforces a consistent ideom across all of them. This commit works, but I'd love advice on how to make linux and darwin better utilize it. @copumpkin I'm especially curious about your feedback, and whether this will help with your goal of automatically solving for bootstrapping stages (I hope it will! but not sure).

The second commit is just the bare-bones of what I want cross-compiling to look like. The big ideas are

• To finally take advantage of bootstrapping rather than have all these *Cross packages and other hacks: "native dependencies" are actually stage n-1 dependencies
• Properly distinguish between host and target in the autotools sense.

I'll probably end up splitting this into two PRs to get the first part merged sooner, but wanted to include the second part for now in order to better motivate the first.

---

Probably closes #14965
Takes a big stab at closing #10874
I shamelessly stole things from #18386 and will steal more!

CC @DavidEGrayson @nbp @shlevy @copumpkin @vcunat

[mention-bot]

@Ericson2314, thanks for your PR! By analyzing the history of the files in this pull request, we identified @edolstra, @errge, @gridaphobe, @LnL7 and @copumpkin to be potential reviewers.

[DavidEGrayson]

Hey, I'm trying out commit ab9eebbba565932ff92acbcb0a1a541119d95468 on my computer right now and it says:

error: undefined variable ‘crossSystem’ at /home/david/nixpkgs/pkgs/top-level/all-packages.nix:4775:20
(use ‘--show-trace’ to show detailed location information)

[Ericson2314]

@DavidEGrayson yeah almost* every crossSystem needs to be triaged into hostPlatform or targetPlatform. There's a lot of work yet to be done.

*I am (for now) keeping crossSystem as the user-facing argument for nixpkgs, and letting the cross stdenv use the *platform args accordingly.

[Ericson2314]

@DavidEGrayson yes everything is broken. I'm making the WIP now to get the basic idea out there. Fixing things is tedious but fairly easy.

[Ericson2314]

@DavidEGrayson Working on GCC 5 now (also where did your comment go, did I delete it by accident or something?!?). How exactly do cross stdenv.cross current map to host vs target? I see both used in places for --target which is...confusing, to say the least

[DavidEGrayson]

I deleted my comment. In my pull request (#18386) I introduced targetCrossSystem and hostCrossSystem variables in the GCC recipe. For a native compiler they are both null. For a cross-compiler, targetCrossSystem will be equal to crossSystem. For a cross-built native compiler, targetCrossSystem and hostCrossSystem will both be equal to crossSystem. There is no support for other combinations yet.

[Ericson2314]

Thanks! that helps a lot

[DavidEGrayson]

Hey, I don't know if your pull request does this yet, but the overrides from the vanilla stdenv should get applied to the final stdenv (the one that can cross-compile things). Otherwise, packages like zlib will change just because the crossSystem argument is supplied. I thought I would mention that because it might have a bearing on the architecture you are making, which I have not really taken the time to understand yet.

[Ericson2314]

@DavidEGrayson I think I am doing that? IIRC the idea is if you traverse enough build-time deps from a crossDrv, one gets back to "build = host = target" (or target doesn't matter) first, and those should be overridden just like when not cross compiling and that's the top bootstrap stage.

[Ericson2314]

On interesting complication is currently we like to build libc in the same faux-stage as gcc/clang. But that's wrong as its a runtime dep. Just as we have stdenvNoCC I think we also need a stdenvNoLibc or similar for this.

[DavidEGrayson]

libc is a build-time dependency because the gcc/clang compiler needs to know where it is so it can compile programs without the user having to specify a libc. I don't understand what is wrong.

[Ericson2314]

@DavidEGrayson so I'm trying to to make "build time dep" ==> "drawn from stage n-1", but naively doign what you say with libc creates a gaping exception because libc must be built for the cross platform. I could have an additional bootstrapping stage for gccStaticCross, but then normal native deps would need to be "n-2" and it gets awkward.

Similarly, while many packages can meaningfully be built with differently-booted regular compilers, packages either do or don't need libc, so if I did crate another bootstrapping stage it ought to be a completely disjoint set of packages---just those which don't need libc. Again, sort of weird.

Finally, I think it's kind of a smell that libc needs to be treated differently than other libraries---in Rust myself and others have been working to try to make the rust standard library built and used as normally as possible. I'd love to ditch the wrapper script and just make libc a default buildInput--libc of course would disable the default to avoid a cycle.

[DavidEGrayson]

I don't understand how the n-1 thing is a good principle; it seems like it would cause you to rebuild some dependencies unnecessarily (e.g. having a zlib at stage k and another one at k+1). My earlier comment is an example where you want to have several dependencies that come from stage n-2. And if we're talking about a normal library and normal program they both would come from the same (final) stage. Also, libc needs to be special because it is part of stdenv.

[Ericson2314]

Ok, I scaled things back leaving the actual hostPlatform targetPlatform rename/refactor as future work. crossSystem vs stdenv.cross is an adequate hack for now to keep the diff down.

[Ericson2314]

OK this is ready to be reviewed!

I haven't tested cross-compiling as much---it ought to have a much better foundation than before, but not necessarily be less broken. I have however tested that linux hashes are unchanged and the travis script passes locally.

Once this is reviewed and merged, I'll do the big crossSystem => hostSystem stdenv.cross => targetSystem rename as a separate PR as it's very big and very boring. I'd like to simultaneously assist with rebasing @DavidEGrayson's work, and use that PR to actually ensure cross compiling works, and is tested so it won't break again.

[Ericson2314]

Heads up, in https://github.com/obsidiansystems/nixpkgs/tree/ghc-android I have a huge number of hash-breaking, but tested improvements. The git history is unclean however. I'm thinking I clean that up and combine it with the few parts of this I haven't already incorporated into this.

After that, I can make a staging which will really clean things up. This will be big, but actually quite simple mainly making most buildPlatform != targetPlatform unconditional true, and with which I think cross compiling can be considered solved 😀.

[FRidh]

@Ericson2314 what should happen with this PR? It hasn't been touched in a long while, and it for sure won't make 19.03.

[Ericson2314]

@FRidh I've basically been keeping it open as a TODO to triage everything in it has either been done or shouldn't be done. Removed the milestone.

[matthewbauer]

syntax

[matthewbauer]

We shouldn't need crossSystem here. Why not just use stdenv.hostPlatform.libc? libc should come from pkgsTargetTarget right?

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[Ericson2314]

Everything in here I think is done or won't be done.

In particular the "wrappers in next stage" idea isn't being done---better to just keep on using targetPackages to get libc etc until #26004 solves the problem for real.