ghc prebuilt: Add 7.8.4, 7.10.3 and 8.2.1, and make consistent style #29688

Ericson2314 · 2017-09-23T01:42:24Z

Motivation for this change

I wanted to keep these maintainable, so I slightly tweaked the style to be consistent everywhere. The new ones have relative RPATHs in ghc-cabal and other build-time utilities.

@peti I do have some of the formatting fixes that you were unsure about before, as part of that normalizing style and methodology, so definitely waiting on your approval.

Now that they are pretty close, we could make a common helper, too.

Closes #19926

Things done

I've currently tested x86_64-linux and x86_64-darwin. Darwin succeeds everywhere. The new Linux ones however don't succeed with:

"utils/ghc-cabal/dist-install/build/tmp/ghc-cabal-bindist" copy libraries/ghc-prim dist-install "strip" '' '/nix/store/y1na8avj8q7jxw74nzi9xd5hmqdb65dl-ghc-7.10.3-binary' '/nix/store/y1na8avj8q7jxw74nzi9xd5hmqdb65dl-ghc-7.10.3-binary/lib/ghc-7.10.3' '/nix/store/y1na8avj8q7jxw74nzi9xd5hmqdb65dl-ghc-7.10.3-binary/share/doc/ghc/html/libraries' 'v p dyn'  
ghc-cabal: Saved package config file body is corrupt. Try re-running the 'configure' command.
make[1]: *** [ghc.mk:924: install_packages] Error 1
make: *** [Makefile:24: install] Error 2

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- Linux
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

CC @cleverca22

mention-bot · 2017-09-23T01:42:27Z

@Ericson2314, thanks for your PR! By analyzing the history of the files in this pull request, we identified @peti, @shlevy and @edolstra to be potential reviewers.

peti

I am all for adding the new builds, and I am completely against modifying the existing builds.

peti · 2017-09-23T14:08:52Z

pkgs/development/compilers/ghc/6.10.2-binary.nix

+{ stdenv
+, fetchurl, perl
+, libedit, ncurses5, gmp
+}:


I don't believe that a consistent style is important for those files and I would prefer to avoid any changes whatsoever in the old compiler builds unless there is a technical reason to make the change. Reformatting whitespace in the argument lists feels pointless to me.

peti · 2017-09-23T14:10:27Z

pkgs/development/compilers/ghc/6.10.2-binary.nix

-        echo compilation ok
-        [ $(./main) == "yes" ]
-      '';
+  dostallCheck = true;


peti · 2017-09-23T15:05:20Z

OK, I now realize why consistent-looking build files are useful in the light of the upcoming cross-compilation changes. I completely revert my previous position and am now in favor of it. :-)

Ericson2314 · 2017-09-23T18:56:41Z

I merged the reformats to master and rebase this (to staging) on top, making it much easier to read.

Ericson2314 · 2017-09-23T18:57:27Z

7.8.4-binary is still failing on Linux from the type of RPATH problem I tried to avoid, however. Quite tempted to just add the missing functionality to patchelf.

These changes do break hashes, but are good to have everywhere for consistency. Plus, changing the bootstrap causes a Haskell mass rebuild anyways.

The newer to-be-added binaries have relative RPATHs that we'll break if we try to `patchelf` before installation. We instead us LD_LOAD_LIBRARY during the installation, which avoids needing to change the RPATH temporarily.

- Patch all executables and libraries, while skipping scripts. Base at least uses libiconv, so should need this too---I suspect it wasn't a problem before as we got away with the immpurity. - Rather than hardcoding the symlinks to add, do one per mach-o as needed TEMP no -L no script

dhess · 2017-10-01T19:28:49Z

This is great, and thank you for including the ARM variants! This is better than the hack I devised to do the same for my ARMv7 and aarch64 systems. I'm looking forward to Hydra building aarch64 Haskell packages so I don't have to build them all myself each time I update!

domenkozar · 2017-10-01T19:34:40Z

@Ericson2314 happy to test this once it's ready. Good work!

dhess · 2017-10-31T07:17:38Z

@Ericson2314 Why not take this PR a step further, and use the binary snapshots you've added to bootstrap the non-binary GHC derivations? (See #19926) You've already done the hard part!

Ericson2314 · 2017-11-03T15:44:20Z

@dhess Well, I haven't completely the hard part because these don't all build! Not sure when I will have time to finish this, so others feel free to push to this.

domenkozar · 2017-12-25T14:02:25Z

@nixborg build

nixborg · 2017-12-25T14:02:27Z

@domenkozar is not a committer

domenkozar · 2017-12-25T14:52:00Z

@Ericson2314 can you explain a bit more what's going on linux? If I understand correctly it's trying to parse /tmp/nix-build-ghc-7.10.3-binary.drv-7/ghc-7.10.3/libraries/ghc-prim/dist-install/setup-config and fails to do so.

domenkozar · 2017-12-25T15:46:31Z

@Ericson2314 following fixes your current error, but resulting $out/bin/ghc still needs patchelfed libs:

$ git diff
diff --git a/pkgs/development/compilers/ghc/7.10.3-binary.nix b/pkgs/development/compilers/ghc/7.10.3-binary.nix
index 1711531fae1..a7a5bd17cfb 100644
--- a/pkgs/development/compilers/ghc/7.10.3-binary.nix
+++ b/pkgs/development/compilers/ghc/7.10.3-binary.nix
@@ -87,9 +87,9 @@ stdenv.mkDerivation rec {

       sed -i "s|/usr/bin/perl|perl\x00        |" ghc-${version}/ghc/stage2/build/tmp/ghc-stage2
       sed -i "s|/usr/bin/gcc|gcc\x00        |" ghc-${version}/ghc/stage2/build/tmp/ghc-stage2
-      for prog in ld ar gcc strip ranlib; do
-        find . -name "setup-config" -exec sed -i "s@/usr/bin/$prog@$(type -p $prog)@g" {} \;
-      done
+      #for prog in ld ar gcc strip ranlib; do
+        #find . -name "setup-config" -exec sed -i "s@/usr/bin/$prog@$(type -p $prog)@g" {} \;
+      #done
     '';

   configurePlatforms = [ ];

domenkozar · 2017-12-25T22:12:24Z

I have all of these building on darwin/linux in #33054

dhess · 2017-12-25T22:56:13Z

Are there plans to add armv7l-linux and aarch64-linux haskellPackages to the Hydra builds?

domenkozar · 2017-12-25T22:57:31Z

@dhess if you want to do the changes, why not :)

dhess · 2017-12-31T18:51:33Z

@domenkozar Is there a reason that ghc821Binary isn't used to bootstrap ghc822 in 695a026, rather than ghc7103Binary?

Unfortunately, because ghc7103Binary doesn't support aarch64-linux, I believe this means we still can't build Haskell packages for aarch64-linux out of the box.

domenkozar · 2017-12-31T18:54:35Z

No good reason, we can bump it.

dhess · 2017-12-31T21:36:15Z

OK, great. I should have time to work on ARM GHC stuff over the next few days, so I'll try it out locally and submit a PR if someone doesn't beat me to it.

dhess · 2018-01-02T12:10:45Z

I just fixed a couple of issues that were preventing GHC from bootstrapping on both ARM platforms. I'm now kicking off armv7l-linux and x86_64-linux builds using ghc821Binary as the bootstrap compiler. If that goes well I'll test aarch64-linux and submit a PR.

Ericson2314 · 2018-01-02T15:02:40Z

Sounds good! But make sure you work from Domen's recent work.

dhess · 2018-01-02T20:40:27Z

I propose we track GHC on ARM issues in #33353.

Ericson2314 added the 2.status: work-in-progress This PR isn't done label Sep 23, 2017

Ericson2314 requested a review from peti as a code owner September 23, 2017 01:42

Ericson2314 added the 6.topic: haskell label Sep 23, 2017

Ericson2314 force-pushed the ghc-prebuilt branch 4 times, most recently from 52cb654 to 17d2fb8 Compare September 23, 2017 02:38

Ericson2314 requested a review from domenkozar September 23, 2017 13:22

peti requested changes Sep 23, 2017

View reviewed changes

peti added the 0.kind: enhancement Add something new label Sep 23, 2017

Ericson2314 force-pushed the ghc-prebuilt branch from fe6dd1d to 086be94 Compare September 23, 2017 14:15

peti approved these changes Sep 23, 2017

View reviewed changes

Ericson2314 changed the base branch from master to staging September 23, 2017 14:57

Ericson2314 force-pushed the ghc-prebuilt branch 4 times, most recently from 1e9d6db to ae243cc Compare September 23, 2017 16:51

Ericson2314 mentioned this pull request Sep 23, 2017

ghc prebuilt: Normalize style without changing hashes #29708

Merged

Ericson2314 changed the title ~~ghc prebuilt: Add 7.8.4 and 7.10.3, and make consistent style~~ ghc prebuilt: Add 7.8.4, 7.10.3 and 8.2.1, and make consistent style Sep 23, 2017

Ericson2314 force-pushed the ghc-prebuilt branch from ae243cc to 429c7e2 Compare September 23, 2017 17:47

Ericson2314 force-pushed the ghc-prebuilt branch from 429c7e2 to bb903e5 Compare September 26, 2017 18:09

Ericson2314 added 4 commits September 26, 2017 16:48

ghc prebuilt: Clean up derivations ever so slightly

123ec88

These changes do break hashes, but are good to have everywhere for consistency. Plus, changing the bootstrap causes a Haskell mass rebuild anyways.

ghc binary: Fix interpreter than patch RPATH

660073e

The newer to-be-added binaries have relative RPATHs that we'll break if we try to `patchelf` before installation. We instead us LD_LOAD_LIBRARY during the installation, which avoids needing to change the RPATH temporarily.

ghc-{7.8.4, 7.10.3, 8.2.1}-binary: Init

960e00d

Ericson2314 force-pushed the ghc-prebuilt branch from bb903e5 to 960e00d Compare September 26, 2017 20:48

dhess mentioned this pull request Oct 1, 2017

Add binary snapshots of GHC versions newer than 7.4.x for quicker bootstrapping #19926

Closed

kalhauge mentioned this pull request Nov 16, 2017

Problem Installing GHC on Raspberry Pi 3 #31666

Closed

domenkozar mentioned this pull request Dec 25, 2017

Use prebuilt GHC to compile hscolor #32609

Open

domenkozar mentioned this pull request Dec 25, 2017

ghc prebuilt: Add 7.8.4, 7.10.3 and 8.2.1, and make consistent style #33054

Merged

domenkozar closed this Dec 25, 2017

Ericson2314 deleted the ghc-prebuilt branch January 1, 2018 23:09

dhess mentioned this pull request Jan 2, 2018

GHC on ARM platforms #33353

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ghc prebuilt: Add 7.8.4, 7.10.3 and 8.2.1, and make consistent style #29688

ghc prebuilt: Add 7.8.4, 7.10.3 and 8.2.1, and make consistent style #29688

Ericson2314 commented Sep 23, 2017 •

edited

Loading

mention-bot commented Sep 23, 2017

peti left a comment

peti Sep 23, 2017

peti Sep 23, 2017

peti commented Sep 23, 2017

Ericson2314 commented Sep 23, 2017

Ericson2314 commented Sep 23, 2017 •

edited

Loading

dhess commented Oct 1, 2017

domenkozar commented Oct 1, 2017

dhess commented Oct 31, 2017 •

edited

Loading

Ericson2314 commented Nov 3, 2017 •

edited

Loading

domenkozar commented Dec 25, 2017

nixborg commented Dec 25, 2017

domenkozar commented Dec 25, 2017

domenkozar commented Dec 25, 2017

domenkozar commented Dec 25, 2017 •

edited

Loading

dhess commented Dec 25, 2017

domenkozar commented Dec 25, 2017

dhess commented Dec 31, 2017 •

edited

Loading

domenkozar commented Dec 31, 2017

dhess commented Dec 31, 2017

dhess commented Jan 2, 2018

Ericson2314 commented Jan 2, 2018

dhess commented Jan 2, 2018

ghc prebuilt: Add 7.8.4, 7.10.3 and 8.2.1, and make consistent style #29688

ghc prebuilt: Add 7.8.4, 7.10.3 and 8.2.1, and make consistent style #29688

Conversation

Ericson2314 commented Sep 23, 2017 • edited Loading

Motivation for this change

Things done

mention-bot commented Sep 23, 2017

peti left a comment

Choose a reason for hiding this comment

peti Sep 23, 2017

Choose a reason for hiding this comment

peti Sep 23, 2017

Choose a reason for hiding this comment

peti commented Sep 23, 2017

Ericson2314 commented Sep 23, 2017

Ericson2314 commented Sep 23, 2017 • edited Loading

dhess commented Oct 1, 2017

domenkozar commented Oct 1, 2017

dhess commented Oct 31, 2017 • edited Loading

Ericson2314 commented Nov 3, 2017 • edited Loading

domenkozar commented Dec 25, 2017

nixborg commented Dec 25, 2017

domenkozar commented Dec 25, 2017

domenkozar commented Dec 25, 2017

domenkozar commented Dec 25, 2017 • edited Loading

dhess commented Dec 25, 2017

domenkozar commented Dec 25, 2017

dhess commented Dec 31, 2017 • edited Loading

domenkozar commented Dec 31, 2017

dhess commented Dec 31, 2017

dhess commented Jan 2, 2018

Ericson2314 commented Jan 2, 2018

dhess commented Jan 2, 2018

Ericson2314 commented Sep 23, 2017 •

edited

Loading

Ericson2314 commented Sep 23, 2017 •

edited

Loading

dhess commented Oct 31, 2017 •

edited

Loading

Ericson2314 commented Nov 3, 2017 •

edited

Loading

domenkozar commented Dec 25, 2017 •

edited

Loading

dhess commented Dec 31, 2017 •

edited

Loading