Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ld on MacOS X 10.9 doesn't accept -E and -rpath options anymore #1157

Closed
nougad opened this issue Nov 1, 2013 · 33 comments
Closed

ld on MacOS X 10.9 doesn't accept -E and -rpath options anymore #1157

nougad opened this issue Nov 1, 2013 · 33 comments
Labels
0.kind: bug Something is broken 6.topic: darwin Running or building packages on Darwin

Comments

@nougad
Copy link

nougad commented Nov 1, 2013

$ nix-env -i ghc-7.6.3
...
building libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.0.0.a
"rm" -f libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.0.0.a libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.0.0.a.contents
echo libraries/ghc-prim/dist-install/build/GHC/Classes.o libraries/ghc-prim/dist-install/build/GHC/CString.o libraries/ghc-prim/dist-install/build/GHC/Debug.o libraries/ghc-prim/dist-install/build/GHC/Magic.o libraries/ghc-prim/dist-install/build/GHC/PrimopWrappers.o libraries/ghc-prim/dist-install/build/GHC/IntWord64.o libraries/ghc-prim/dist-install/build/GHC/Tuple.o libraries/ghc-prim/dist-install/build/GHC/Types.o   libraries/ghc-prim/dist-install/build/cbits/debug.o libraries/ghc-prim/dist-install/build/cbits/longlong.o libraries/ghc-prim/dist-install/build/cbits/popcnt.o     >> libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.0.0.a.contents
"xargs"  "/nix/store/3y3300pb3zwywb7qh5mzvxp1gf7za8l1-native-darwin-cctools-wrapper/bin/ar" clqs  libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.0.0.a < libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.0.0.a.contents
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.0.0.a(longlong.o) has no symbols
"rm" -f libraries/ghc-prim/dist-install/build/libHSghc-prim-0.3.0.0.a.contents
building libraries/ghc-prim/dist-install/build/HSghc-prim-0.3.0.0.o
  LD libraries/ghc-prim/dist-install/build/HSghc-prim-0.3.0.0.o
ld: -rpath can only be used when targeting Mac OS X 10.5 or later
make[1]: *** [libraries/ghc-prim/dist-install/build/HSghc-prim-0.3.0.0.o] Error 1
make: *** [all] Error 2
builder for `/nix/store/wy8yjzihrfykr7lssngl2sfa051g975f-ghc-7.6.3.drv' failed with exit code 2
error: build of `/nix/store/wy8yjzihrfykr7lssngl2sfa051g975f-ghc-7.6.3.drv' failed

when I export MACOSX_DEPLOYMENT_TARGET=10.9 then ld doesn't complains about -rpath but it seams that nix deletes this variable before building ghc. So even if I set the variable before I get the same error.

@peti
Copy link
Member

peti commented Nov 2, 2013

Could you please verify that the error is gone after applying this patch to your local copy of Nixpkgs?

@peti
Copy link
Member

peti commented Nov 2, 2013

Does this error occur with GHC only? There should be plenty of builds in Nixpkgs that use the -rpath flag, so I guess all those builds would have to run into this issue, right?

@nougad
Copy link
Author

nougad commented Nov 2, 2013

Sorry, I'm completely new to nix. So I only installed a few packages. GHC-7.6.3 and lua-5.2.2 fails for me.

I tried to apply the patch but still fails:

$ cd /nix/store/mxl4lhgzgvc65fgah7mk0aaq17y1pyc1-nixpkgs-13.10pre35413.86443f4/nixpkgs/pkgs/development/compilers/ghc
$ patch < /tmp/da09c4043a53e351f2956912008f0737423d0e91.patch
patching file 7.4.2.nix
patching file 7.6.3.nix
patching file head.nix
$ nix-env -i ghc-7.6.3
...
  LD libraries/ghc-prim/dist-install/build/HSghc-prim-0.3.0.0.o
ld: -rpath can only be used when creating a dynamic final linked image
make[1]: *** [libraries/ghc-prim/dist-install/build/HSghc-prim-0.3.0.0.o] Error 1

problem for lua-5.2.2:

$ nix-env -i lua-5.2.2
...
building lua
gcc -o lua   lua.o liblua.a -lm -Wl,-E -ldl -lreadline
ld: unknown option: -E
collect2: ld returned 1 exit status

@peti
Copy link
Member

peti commented Nov 2, 2013

Something about your MacOS machine is different from the boxes we normally use to build Nixpkgs. Neither of these errors occurs when Hydra builds ghc or lua. I'm not a Darwin expert, I'm afraid, so I can't help much resolving this.

@nougad
Copy link
Author

nougad commented Nov 2, 2013

I run newest Mac os x. I have installed brew in parallel with gcc/ghc/.. but I take care that this doesn't affect nix. I tried to delete whole nix but no success. Build still fails.

I have no idea what I can try. I think I rebuild ghc 20 times with all modifications. But no idea. Anyway thanks for your help. If you have no problems then I's maybe me environment and you can close the ticket.

@peti
Copy link
Member

peti commented Nov 4, 2013

I'm just speculating (I don't access to Darwin), but my guess is that the latest MacOS X changed in a way that affects those Nix builds, i.e. by using different a different ld. If that is true, then Nix needs to address this issue somehow. I wouldn't close this ticket until someone has come up with some reasonable explanation of why these link errors occur.

@vcunat
Copy link
Member

vcunat commented Nov 5, 2013

We seem to have this problem on Hydra already http://hydra.nixos.org/build/6731928

@vcunat
Copy link
Member

vcunat commented Nov 5, 2013

Well, the error looks a bit different, but I suppose it's related.

@nougad
Copy link
Author

nougad commented Nov 5, 2013

No, this looks exactly the same as my error. See my second comment on this bug.

@vcunat
Copy link
Member

vcunat commented Nov 5, 2013

Ah, I see, that's probably because the build is after @peti pushed a patch -- it changed a little.

@peti
Copy link
Member

peti commented Nov 6, 2013

@nougad, my patch adds the shell command

export MACOSX_DEPLOYMENT_TARGET=10.5

to the GHC build, but apparently that isn't good enough. Could you please edit the patched file pkgs/development/compilers/ghc/7.6.3.nix so that it says

export MACOSX_DEPLOYMENT_TARGET=10.9

instead? Maybe that improves matters?

@nougad
Copy link
Author

nougad commented Nov 6, 2013

nope. =10.9 leads to the same error:

ld: -rpath can only be used when creating a dynamic final linked image

@peti
Copy link
Member

peti commented Nov 6, 2013

OK. I have a new idea. Could you please apply this patch to a fresh version of Nixpkgs and try building GHC one more time?

In case the build does not succeed, please look for a line

checking Mac OS X deployment target... 

in the ./configure output from GHC. What does that line say?

@nougad
Copy link
Author

nougad commented Nov 6, 2013

Sorry, it seams that I doing something wrong:

$ nix-env -i ghc-7.6.3
...
configuring
configure flags: --prefix=/nix/store/r6gav3xa0z2ji1is6pa8gxzz0kgcd8gs-ghc-7.6.3 --with-gcc=/nix/store/smk3x4shm3c9s6cz263qhkdjy8yinkcb-gcc-wrapper-4.6.3/bin/gcc --macosx-deployment-target=10.5
configure: error: unrecognized option: `--macosx-deployment-target=10.5'
Try `./configure --help' for more information

@peti
Copy link
Member

peti commented Nov 6, 2013

Oops, my mistake. I misspelled the name of the configure option. Here is a corrected patch that should work better. Sorry!

@nougad
Copy link
Author

nougad commented Nov 6, 2013

nix-env -i ghc-7.6.3
warning: there are multiple derivations named `ghc-7.6.3'; using the first one
installing `ghc-7.6.3'
these derivations will be built:
  /nix/store/qnm2ryynrx65kkpxqx5fjg7xrj9h75b8-ghc-7.6.3.drv
building path(s) `/nix/store/lw8kdnz7m1b2xwsbqdr19b69098zll09-ghc-7.6.3'
building /nix/store/lw8kdnz7m1b2xwsbqdr19b69098zll09-ghc-7.6.3
unpacking sources
unpacking source archive /nix/store/49skjzfpb5vzp4qxaqp0nyirp67hryxy-ghc-7.6.3-src.tar.bz2
source root is ghc-7.6.3
patching sources
configuring
configure flags: --prefix=/nix/store/lw8kdnz7m1b2xwsbqdr19b69098zll09-ghc-7.6.3 --with-gcc=/nix/store/smk3x4shm3c9s6cz263qhkdjy8yinkcb-gcc-wrapper-4.6.3/bin/gcc --with-macosx-deployment-target=10.5
checking for gfind... no
checking for find... /nix/store/q8q92lpcbqxf1sdbdgiqp79klllm26h2-findutils-4.4.2/bin/find
checking for sort... /nix/store/yq1a2ajx4igi7sr8wv8scr36kvp8607f-coreutils-8.21/bin/sort
checking for ghc... /nix/store/zrpm8a6by1j4kfcp0yg1apqnd7dfdxfr-ghc-7.0.4-binary/bin/ghc
checking version of ghc... 7.0.4
checking build system type... x86_64-apple-darwin13.0.0
checking host system type... x86_64-apple-darwin13.0.0
checking target system type... x86_64-apple-darwin13.0.0
HOST: x86_64-apple-darwin13.0.0
Build platform inferred as: x86_64-apple-darwin
Host platform inferred as: x86_64-apple-darwin
Target platform inferred as: x86_64-apple-darwin
GHC build  : x86_64-apple-darwin
GHC host   : x86_64-apple-darwin
GHC target : x86_64-apple-darwin
configure: Building in-tree ghc-pwd
ld: warning: PIE disabled. Absolute addressing (perhaps -mdynamic-no-pic) not allowed in code signed PIE, but used in ___gmpn_modexact_1c_odd from /nix/store/zrpm8a6by1j4kfcp0yg1apqnd7dfdxfr-ghc-7.0.4-binary/lib/ghc-7.0.4/integer-gmp-0.2.0.3/libHSinteger-gmp-0.2.0.3.a(mode1o.o). To fix this warning, don't compile with -mdynamic-no-pic or link with -Wl,-no_pie
checking for path to top of build tree... /private/var/folders/fc/f1bwrr4n05qfb430d5qndg8w0000gn/T/nix-build-ghc-7.6.3.drv-0/ghc-7.6.3
checking XCode version... ./configure: line 3990: xcodebuild: command not found
not found (too old?)
checking for ld... /nix/store/smk3x4shm3c9s6cz263qhkdjy8yinkcb-gcc-wrapper-4.6.3/bin/ld
checking for nm... /nix/store/3y3300pb3zwywb7qh5mzvxp1gf7za8l1-native-darwin-cctools-wrapper/bin/nm
checking for llc... no
checking for opt... no
checking Mac OS X deployment target... configure: error: Unknown deployment target 10.5
builder for `/nix/store/qnm2ryynrx65kkpxqx5fjg7xrj9h75b8-ghc-7.6.3.drv' failed with exit code 1
error: build of `/nix/store/qnm2ryynrx65kkpxqx5fjg7xrj9h75b8-ghc-7.6.3.drv' failed

The xcodebuild: command not found is strage because it is in the path:

$ which xcodebuild
/usr/bin/xcodebuild

$ cd /private/var/folders/fc/f1bwrr4n05qfb430d5qndg8w0000gn/T/nix-build-ghc-7.6.3.drv-0/ghc-7.6.3
$ xcodebuild -version | grep Xcode | sed "s/Xcode //" # see configure
5.0.1

I changed the --with-macosx-deployment-target=10.5 to 10.9 but same result. Because the ./configure looks for: /Developer/SDKs/MacOSX${FP_MACOSX_DEPLOYMENT_TARGET}.sdk. But it is located at /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk. I created a symlink cd /; sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer ..

But then I get again the error

LD libraries/ghc-prim/dist-install/build/HSghc-prim-0.3.0.0.o
ld: -rpath can only be used when targeting Mac OS X 10.5 or later

So I added again:

+  '' + stdenv.lib.optionalString stdenv.isDarwin ''
+      export MACOSX_DEPLOYMENT_TARGET=10.5

But this leads to the same error again:

LD libraries/ghc-prim/dist-install/build/HSghc-prim-0.3.0.0.o
ld: -rpath can only be used when creating a dynamic final linked image

@peti
Copy link
Member

peti commented Nov 7, 2013

My guess is that this build won't succeed as long as ./configure says

checking Mac OS X deployment target... configure: error: Unknown deployment target 10.5

I'm not sure why it won't accept that deployment target. Maybe someone else knows?

Anyway, /usr/bin is not in $PATH when those builds are run. This is why xcodebuild cannot be found. I'm not sure whether this is an issue; I have no idea what xcodebuild is, I'm afraid.

@peti
Copy link
Member

peti commented Nov 7, 2013

Hmm, maybe b579f4f is the reason why the deployment target isn't recognized ...

I'm beginning to feel like the best way to deal with this situation for now is to just remove the use of -rpath. :-(

@nougad, please try this patch. That one should fix the build for good, possibly at the expense of breaking shared library support, though.

@nougad
Copy link
Author

nougad commented Nov 7, 2013

This line only occurs without the symlink I made. If I create the symlink it says:

checking Mac OS X deployment target... 10.9 (/Developer/SDKs/MacOSX10.9.sdk)

But the build still fails with the error.

xcodebuild is for determining the xcode version and doing workarounds for <3.2 and version 4.2. But due xcodebuild binary is not found the version is set to 0.0 which is < 3.2 and so maybe this leads to an error? I don't know what SplitObjsBroken=YES is doing.

@peti
Copy link
Member

peti commented Nov 7, 2013

Well, you can make xcodebuild available to the build by adding to the list of required native binaries in pkgs/build-support/native-darwin-cctools-wrapper/default.nix. Doing this will probably trigger massive re-builds, though; it's a fairly expensive change to make.

The SplitObjs option is a nice optimization that splits Haskell modules into one .o object per function internally before building the corresponding lib.a file. This means that the linker can link every function individually, instead of having to link the entire module. Having this enabled oftentimes results is significantly smaller binaries, so having that feature "unbroken" would certainly be worthwhile.

@nougad
Copy link
Author

nougad commented Nov 7, 2013

Ok I give up. I switch back to homebrew. Nix is not made for osx.

I tried the following combinations:

build nix-env -i ghc-7.6.3

symlink: : cd /; sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer .
xcodebuild: in pkgs/build-support/native-darwin-cctools-wrapper/default.nix

  • unpatched
    -> ld: -rpath can only be used when targeting Mac OS X 10.5 or later
  • patch da09c4043 export MACOSX_DEPLOYMENT_TARGET=10.5
    -> ld: -rpath can only be used when creating a dynamic final linked image
  • patch da09c4043 + 10.9 export MACOSX_DEPLOYMENT_TARGET=10.9
    -> ld: -rpath can only be used when creating a dynamic final linked image
  • patch 9929a94c4 --with-macosx-deployment-target=10.5
    -> checking XCode version... ./configure: line 3990: xcodebuild: command not found
    -> checking Mac OS X deployment target... configure: error: Unknown deployment target 10.5
    -> ld: -rpath can only be used when creating a dynamic final linked image
  • patch 9929a94c4 + 10.9 --with-macosx-deployment-target=10.9
    -> checking XCode version... ./configure: line 3990: xcodebuild: command not found
    -> checking Mac OS X deployment target... configure: error: Unknown deployment target 10.9
    -> ld: -rpath can only be used when creating a dynamic final linked image
  • patch 9929a94c4 + 10.9 + symlink
    -> checking XCode version... ./configure: line 3990: xcodebuild: command not found
    -> checking Mac OS X deployment target... 10.9 (/Developer/SDKs/MacOSX10.9.sdk)
    -> ld: -rpath can only be used when creating a dynamic final linked image
  • patch 9929a94c4 + 10.9 + symlink + (da09c4043 + 10.9)
    -> checking XCode version... ./configure: line 3990: xcodebuild: command not found
    -> checking Mac OS X deployment target... 10.9 (/Developer/SDKs/MacOSX10.9.sdk)
    -> ld: -rpath can only be used when creating a dynamic final linked image
  • patch 5078ac97d + symlink
    -> checking XCode version... ./configure: line 3990: xcodebuild: command not found
    -> checking Mac OS X deployment target... none
    -> ld: -rpath can only be used when targeting Mac OS X 10.5 or later
  • patch 5078ac97d + (9929a94c4 + 10.9) + symlink
    -> checking XCode version... ./configure: line 3990: xcodebuild: command not found
    -> checking Mac OS X deployment target... none
    -> ld: -rpath can only be used when targeting Mac OS X 10.5 or later
  • patch 5078ac97d + (da09c4043 + 10.9) + symlink
    -> checking XCode version... ./configure: line 3990: xcodebuild: command not found
    -> checking Mac OS X deployment target... none
    -> ld: -rpath can only be used when targeting Mac OS X 10.5 or later
  • (da09c4043 + 10.9) + symlink + xcodebuild
  • (patch 9929a94c4 + 10.9) + symlink + (da09c4043 + 10.9) + xcodebuild

But to change native-darwin-cctools-wrapper I needed to delete my complete /nix/store (it was only for testing purpose). But after install whole nix from scratch I'm not able anymore to build gmp-4.3.2 required for ghc:

creating t-sub
make[4]: Leaving directory `/private/var/folders/fc/f1bwrr4n05qfb430d5qndg8w0000gn/T/nix-build-gmp-4.3.2.drv-0/gmp-4.3.2/tests'
make  check-TESTS
make[4]: Entering directory `/private/var/folders/fc/f1bwrr4n05qfb430d5qndg8w0000gn/T/nix-build-gmp-4.3.2.drv-0/gmp-4.3.2/tests'
building check-TESTS
/bin/sh: line 4: 14535 Segmentation fault: 11  ${dir}$tst
FAIL: t-bswap
PASS: t-constants
PASS: t-count_zeros
PASS: t-gmpmax
PASS: t-hightomask
PASS: t-modlinv
/bin/sh: line 4: 14646 Segmentation fault: 11  ${dir}$tst
FAIL: t-popc
PASS: t-parity
PASS: t-sub
====================================
2 of 9 tests failed
Please report to [email protected]
====================================
make[4]: *** [check-TESTS] Error 1
make[4]: Leaving directory `/private/var/folders/fc/f1bwrr4n05qfb430d5qndg8w0000gn/T/nix-build-gmp-4.3.2.drv-0/gmp-4.3.2/tests'
make[3]: *** [check-am] Error 2
make[3]: Leaving directory `/private/var/folders/fc/f1bwrr4n05qfb430d5qndg8w0000gn/T/nix-build-gmp-4.3.2.drv-0/gmp-4.3.2/tests'
make[2]: *** [check-recursive] Error 1
make[2]: Leaving directory `/private/var/folders/fc/f1bwrr4n05qfb430d5qndg8w0000gn/T/nix-build-gmp-4.3.2.drv-0/gmp-4.3.2/tests'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/private/var/folders/fc/f1bwrr4n05qfb430d5qndg8w0000gn/T/nix-build-gmp-4.3.2.drv-0/gmp-4.3.2'
make: *** [check] Error 2
builder for `/nix/store/lk2b83ain5g17yiqx36yrrc5zicmwss8-gmp-4.3.2.drv' failed with exit code 2
cannot build derivation `/nix/store/5am5zlpnv46wiqwazc879sb6s51a70j5-ghc-7.0.4-binary.drv': 1 dependencies couldn't be built
cannot build derivation `/nix/store/rywbrhrvvzpr1pzrkd39h82iv4s7zjjn-ghc-7.6.3.drv': 1 dependencies couldn't be built
error: build of `/nix/store/rywbrhrvvzpr1pzrkd39h82iv4s7zjjn-ghc-7.6.3.drv' failed

So I can't test the last two combinations with enabled xcodebuild.

@vcunat
Copy link
Member

vcunat commented Nov 7, 2013

Well, there's been very little effort to make things work on MacOS, that's IMO the main reason. On homebrew there was surely much work done to make stuff work OK. Most of these tweaks should be directly portable to nix, so it's not like starting all over, but someone would have to do it first.

AFAIK almost none of the most active contributors own/use a Mac, so it's very difficult for them to fix things -- they only have Hydra for detecting some build problems, with a prohibitive latency and cluttering the repo history.

@nougad
Copy link
Author

nougad commented Nov 7, 2013

Sorry if this sounded offending. I really thank you for your help! I like the idea of nix a lot. It is a way better approach then brew. Especially all the haskell packages are interesting for me. But I think I tried compiling ghc (and other packages for example lua as well) about 30 times without any success. This is not your fault. The new mac os x is only a few weeks old and not everything is ported. I can fully understand this. Maybe I try it in the future again.

@vcunat
Copy link
Member

vcunat commented Nov 7, 2013

I did not take it as offending -- I was just explaining my view why it's in this state. But a few people does seem to use nix on Macs lately, submitting pull requests, so I hope it's slowly getting better.

@wavewave
Copy link
Contributor

wavewave commented Jan 9, 2014

@nougad, I made gmp compliation work after including xcodebuild in native-darwin-cctools-wrapper.

f5a7212
58f5dff

@nougad
Copy link
Author

nougad commented Jan 10, 2014

Sorry, I currently have no access to any mac os x. I will try it again when I got a mac again.

@wavewave
Copy link
Contributor

@nougad @peti @vcunat : I made ghc-7.6.3 buildable by hard-fixing mac os x platform to 10.9. Also, I remove additional -rpath to ghc path in NIX_LD_FLAGS which seems to be not required. At least, on my system (10.8 with Xcode 5), ghc and ghci seems to work.

a22fa75

This is based on f5a7212 and 58f5dff

@wavewave
Copy link
Contributor

I found that ghc is buildable without xcodebuild patch in stdenv if I applied the previous patches. See d8eb6cb with which I installed ghc. I am going to make a pull request for ghc on darwin soon based on these patches.

@peti
Copy link
Member

peti commented Jan 17, 2014

@wavewave, it would be great if you could submit a pull request that contains exactly those changes required to compile GHC on Darwin, but nothing else. Then it would be easier for other people to take a look at the changes.

@wavewave
Copy link
Contributor

@peti I made a squashed pull request for this, which annihilated xcodebuild change in native-darwin-cctools-wrapper (i.e. no change there. )

@shlevy shlevy added the darwin label Apr 5, 2014
@shlevy
Copy link
Member

shlevy commented Apr 5, 2014

Is this still broken?

@peti
Copy link
Member

peti commented Apr 5, 2014

The original issue with GHC is fixed. I don't know about other packages that might run into this problem.

@shlevy
Copy link
Member

shlevy commented Apr 5, 2014

Closing until a known issue pops up.

@shlevy shlevy closed this as completed Apr 5, 2014
CodeRadu pushed a commit to CodeRadu/nixpkgs that referenced this issue Feb 25, 2025
Co-authored-by: Milo123459 <[email protected]>
Co-authored-by: Jake Runzer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: bug Something is broken 6.topic: darwin Running or building packages on Darwin
Projects
None yet
Development

No branches or pull requests

5 participants