cctools-port 949.0.1 broke various builds on macOS #101330

r-burns · 2020-10-22T07:35:54Z

Describe the bug
Ruby configure step has been failing on darwin since 3c6bd61 with the following error:

checking whether CFLAGS is valid... yes
checking whether LDFLAGS is valid... no
configure: error: something wrong with LDFLAGS=""

This appears to be due to the new linker warning ld: warning: passed two min versions (10.12.0, 10.12) for platform macOS, as adding LDFLAGS="-Wl,-w" to ruby allows it to build successfully.

git bisect log:

git bisect start
# bad: [92c96b799528442f6799f829d11602786689b28e] pkgs-lib/tests/formats: fix whitespace after PR #100372
git bisect bad 92c96b799528442f6799f829d11602786689b28e
# good: [53ce0bfc904a8e0dbe0d81cc1624815d4d8a038f] Merge pull request #95592 from risicle/ris-flink-1.11.1
git bisect good 53ce0bfc904a8e0dbe0d81cc1624815d4d8a038f
# good: [4c6dad5e826929ecd125472b4608b493b3efc24e] lsd: add maintainer
git bisect good 4c6dad5e826929ecd125472b4608b493b3efc24e
# good: [6d174dd5e0650073971a75efec218c79cbce951f] linux: Remove 5.7
git bisect good 6d174dd5e0650073971a75efec218c79cbce951f
# bad: [27143f8e022cbc9e0a480c3b89d757920aef55f6] nodejs-12_x: 12.18.4 -> 12.19.0
git bisect bad 27143f8e022cbc9e0a480c3b89d757920aef55f6
# bad: [0a5be167a3b5808c2b42a8660cf015c79965b204] Merge pull request #97098 from volth/cpan2nix-2020-09-04
git bisect bad 0a5be167a3b5808c2b42a8660cf015c79965b204
# good: [e9f7e3a97350133bdcd699ef44fbbf9d5678968b] [cpan2nix] perlPackages.PPIxRegexp: 0.072 -> 0.074
git bisect good e9f7e3a97350133bdcd699ef44fbbf9d5678968b
# good: [f728bea688227536a73b382ec1a2afb8be2854d0] [cpan2nix] perlPackages.CatalystViewTT: 0.44 -> 0.45
git bisect good f728bea688227536a73b382ec1a2afb8be2854d0
# bad: [5e68d203f3f8abf8f928fb6788487b83e3a0e8ec] opencv2: mark as broken
git bisect bad 5e68d203f3f8abf8f928fb6788487b83e3a0e8ec
# bad: [9112a9d882ed58d1823db8558d0bc8ceb2f33a5e] Merge pull request #93596 from obsidiansystems/ios-13
git bisect bad 9112a9d882ed58d1823db8558d0bc8ceb2f33a5e
# good: [1ca7c6e76a5138f7f0f3661c4d372fe15c0905c2] Merge pull request #97485 from marsam/update-nodejs-14_x
git bisect good 1ca7c6e76a5138f7f0f3661c4d372fe15c0905c2
# bad: [95eabdfd5f6197c83e9e5f53ddb65dbbea3c13fd] xcode/sdk-pkgs.nix: set -platform_version in addition to -miphoneos-version-min
git bisect bad 95eabdfd5f6197c83e9e5f53ddb65dbbea3c13fd
# bad: [3c6bd61560d042ce1e366989b92b615dfa1dffb3] darwin.cctools: 927.0.2 -> 949.0.1
git bisect bad 3c6bd61560d042ce1e366989b92b615dfa1dffb3
# good: [39c536c8db4e0982f89b67a194cd36ba9f4fbdd5] darwin.libtapi: update, fix installTargets
git bisect good 39c536c8db4e0982f89b67a194cd36ba9f4fbdd5
# first bad commit: [3c6bd61560d042ce1e366989b92b615dfa1dffb3] darwin.cctools: 927.0.2 -> 949.0.1

~~Further bisection on the cctools-port repo itself is unproductive, simply leading to the mass update tpoechtrager/cctools-port@466063c~~ See #101330 (comment) for the offending code in cctools.

Hydra build
https://hydra.nixos.org/build/128219205

Notify maintainers
cctools: @matthewbauer
ruby: @vrthra @manveru @marsam

The text was updated successfully, but these errors were encountered:

SuperSandro2000 · 2020-10-24T11:28:57Z

Diff if someone needs it quickly.

I couldn't find the root cause and it is breaking a lot for darwin so we maybe should just take this fix.

diff --git a/pkgs/development/interpreters/ruby/default.nix b/pkgs/development/interpreters/ruby/default.nix
index 1fa5ea0aecfb..8fbbdefb1afc 100644
--- a/pkgs/development/interpreters/ruby/default.nix
+++ b/pkgs/development/interpreters/ruby/default.nix
@@ -126,6 +126,7 @@ let
             "--with-out-ext=tk"
             # on yosemite, "generating encdb.h" will hang for a very long time without this flag
             "--with-setjmp-type=setjmp"
+            # fix error after upgrading cctools-port to 949.0.1
+            # https://github.com/NixOS/nixpkgs/issues/101330
+            "LDFLAGS=-Wl,-w"
           ]
           ++ op (stdenv.hostPlatform != stdenv.buildPlatform)
              "--with-baseruby=${buildRuby}";

r-burns · 2020-10-24T19:35:37Z

@jonringer I think this is also causing haskellPackages.trifecta to fail which is blocking the unstable channel 👀 https://hydra.nixos.org/build/129070963/nixlog/1

It seems to me that there is a deeper issue here involving the new cctools and how they are being invoked:

Why are two min versions being passed? Can we avoid that?
If not, is it feasible going forward to keep patching this warning out of cctools?

r-burns · 2020-10-24T20:13:44Z

I think the correct ping is @NixOS/darwin-maintainers but I don't seem to have the clearance ;)

zowoq · 2020-10-24T21:59:04Z

cc @NixOS/darwin-maintainers

r-burns · 2020-10-24T23:23:23Z

I'm currently testing out this patch to disable the warning - ruby is working, haskellPackages.trifecta ~~still building~~ also works. Not sure if this is the right solution though, if the multiple versions being passed in the first place are the root problem.

diff --git a/cctools/ld64/src/ld/Options.cpp b/cctools/ld64/src/ld/Options.cpp
index 5b8c755..1b8ebf8 100644
--- a/cctools/ld64/src/ld/Options.cpp
+++ b/cctools/ld64/src/ld/Options.cpp
@@ -3187,8 +3187,6 @@ void Options::parse(int argc, const char* argv[])
 				}
 				if (fPlatforms.contains(info->platform)) {
 					std::string existingVersionStr = getVersionString32(fPlatforms.minOS(info->platform));
-					warning("passed two min versions (%s, %s) for platform %s. Using %s.",
-							existingVersionStr.c_str(), versStr, info->printName, versStr);
 					fPlatforms.updateMinVersion(info->platform, minVersValue);
 					fPlatforms.updateSDKVersion(info->platform, SDKValue);
 				} else {
@@ -3210,8 +3208,6 @@ void Options::parse(int argc, const char* argv[])
 				}
 				if (fPlatforms.contains(info->platform)) {
 					std::string existingVersionStr = getVersionString32(fPlatforms.minOS(info->platform));
-					warning("passed two min versions (%s, %s) for platform %s. Using %s.",
-							existingVersionStr.c_str(), versStr, info->printName, versStr);
 					fPlatforms.updateMinVersion(info->platform, value);
 				} else {
 					fPlatforms.insert(ld::PlatformVersion(info->platform, value));

zowoq · 2020-10-24T23:49:20Z

Seems that the root cause is mentioned in the cctools PR: #93596 (comment)

r-burns · 2020-10-25T00:07:07Z

I guess I'm still not really clear on who's in the wrong here. Is cctools being overly strict by adding this warning? Or is clang overstepping its role by auto-adding extra flags?

zowoq · 2020-10-25T00:20:25Z

I think #93596 (comment) means that this is the problem ~~now that~~ as clang is also adding the args:

nixpkgs/pkgs/build-support/bintools-wrapper/default.nix

Line 255 in 61d82f0

    
                 echo "-macosx_version_min 10.12 -sdk_version 10.12 -no_uuid" >> $out/nix-support/libc-ldflags-before

zowoq · 2020-10-25T01:56:12Z

oops forgot the ping

cc @thefloweringash

r-burns · 2020-10-25T02:32:16Z

Ok, here's a patch to skip clang's addMinVersionArgs. Ruby is working under this as well. Haven't gotten to trifecta yet but probably works.

diff --git a/pkgs/development/compilers/llvm/7/clang/default.nix b/pkgs/development/compilers/llvm/7/clang/default.nix
index 905ec862ad7..c5df66bd983 100644
--- a/pkgs/development/compilers/llvm/7/clang/default.nix
+++ b/pkgs/development/compilers/llvm/7/clang/default.nix
@@ -43,6 +43,7 @@ let
       ./purity.patch
       # make clang -xhip use $PATH to find executables
       ./HIP-use-PATH-7.patch
+      ./dont-add-args.patch
     ];
 
     postPatch = ''
diff --git a/pkgs/development/compilers/llvm/7/clang/dont-add-args.patch b/pkgs/development/compilers/llvm/7/clang/dont-add-args.patch
new file mode 100644
index 00000000000..f164ebe1b73
--- /dev/null
+++ b/pkgs/development/compilers/llvm/7/clang/dont-add-args.patch
@@ -0,0 +1,14 @@
+diff --git a/lib/Driver/ToolChains/Darwin.cpp b/lib/Driver/ToolChains/Darwin.cpp
+index dc540688d2b..1f73c81121c 100644
+--- a/lib/Driver/ToolChains/Darwin.cpp
++++ b/lib/Driver/ToolChains/Darwin.cpp
+@@ -326,9 +326,6 @@ void darwin::Linker::AddLinkArgs(Compilation &C, const ArgList &Args,
+   Args.AddAllArgs(CmdArgs, options::OPT_image__base);
+   Args.AddAllArgs(CmdArgs, options::OPT_init);
+ 
+-  // Add the deployment target.
+-  MachOTC.addMinVersionArgs(Args, CmdArgs);
+-
+   Args.AddLastArg(CmdArgs, options::OPT_nomultidefs);
+   Args.AddLastArg(CmdArgs, options::OPT_multi__module);
+   Args.AddLastArg(CmdArgs, options::OPT_single__module);

zowoq · 2020-10-25T08:04:22Z

May as well keep this open until the main problem is resolved.

r-burns · 2020-10-25T17:29:31Z

Another option that seems to work: removing our extra -macosx_version_min from bintools-wrapper since it is already added by clang. Builds ruby and haskellPackages.trifecta.

diff --git a/pkgs/build-support/bintools-wrapper/default.nix b/pkgs/build-support/bintools-wrapper/default.nix
index 786f0f9c598..a3f1f54d5c5 100644
--- a/pkgs/build-support/bintools-wrapper/default.nix
+++ b/pkgs/build-support/bintools-wrapper/default.nix
@@ -252,7 +252,7 @@ stdenv.mkDerivation {
 
     # Ensure consistent LC_VERSION_MIN_MACOSX and remove LC_UUID.
     + optionalString stdenv.targetPlatform.isMacOS ''
-      echo "-macosx_version_min 10.12 -sdk_version 10.12 -no_uuid" >> $out/nix-support/libc-ldflags-before
+      echo "-sdk_version 10.12 -no_uuid" >> $out/nix-support/libc-ldflags-before
     ''
 
     ##

So to recap, the options that have been tested to work so far are

remove warning from cctools
remove extra min-version arg from clang
remove extra min-version arg from our bintools wrapper

SuperSandro2000 · 2020-10-26T18:41:08Z

Also encountering this issue when building pyobjc. ~~their test suite thinks this is fatal and fails.~~

SuperSandro2000 · 2020-10-26T19:06:04Z

I tested the bintool option which caused a rebuilt of literally everything. I think this is maybe the best approach in the long term but it is totally unsuitable for local rebuilds.

Next I tried the cctools patch which only caused the Darwin Frameworks, perl and some others to rebuild which could work locally. Same with the clang.

zowoq · 2020-10-29T00:20:30Z

Pinging @NixOS/darwin-maintainers again, also @Ericson2314 who may be able to help?

zowoq · 2020-11-07T05:00:25Z

Opened a PR: #103053

glittershark · 2020-11-12T20:18:43Z

This is also breaking gdal

zowoq · 2020-11-13T21:15:10Z

Merged #103053 to staging, I'll keep this open until it lands in master as #101602 needs to be reverted.

zowoq · 2020-11-21T22:38:02Z

Reverting #101602 in #104535.

r-burns added the 0.kind: bug Something is broken label Oct 22, 2020

veprbl added 6.topic: darwin Running or building packages on Darwin 6.topic: ruby labels Oct 22, 2020

marsam mentioned this issue Oct 23, 2020

ruby: fix build on darwin #101441

Merged

10 tasks

r-burns mentioned this issue Oct 24, 2020

nuspell: 3.1.2 -> 4.0.0, fix license #101512

Merged

10 tasks

zowoq mentioned this issue Oct 24, 2020

haskellPackages.trifecta: dontCheck on darwin #101602

Merged

marsam closed this as completed in #101441 Oct 25, 2020

zowoq reopened this Oct 25, 2020

r-burns changed the title ~~cctools-port 949.0.1 broke ruby build on macOS~~ cctools-port 949.0.1 broke various builds on macOS Oct 26, 2020

punkeel mentioned this issue Oct 27, 2020

[Darwin] ruby error on ripgrep install #101877

Closed

brendanhay mentioned this issue Oct 28, 2020

build: migrate to hercules ci, haskell.nix, and upgrade to ghc 8.8.4 urbit/urbit#3806

Merged

zowoq mentioned this issue Nov 7, 2020

bintools-wrapper: remove redundant arg #103053

Merged

10 tasks

thefloweringash mentioned this issue Nov 9, 2020

darwin (Big Sur): tbd based stdenv #98541

Merged

14 tasks

zowoq self-assigned this Nov 13, 2020

zowoq closed this as completed Nov 21, 2020

zowoq removed their assignment Nov 21, 2020

bobrik mentioned this issue Jan 31, 2021

Native support for Apple Silicon #105026

Merged

21 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cctools-port 949.0.1 broke various builds on macOS #101330

cctools-port 949.0.1 broke various builds on macOS #101330

r-burns commented Oct 22, 2020 •

edited

Loading

SuperSandro2000 commented Oct 24, 2020 •

edited

Loading

r-burns commented Oct 24, 2020

r-burns commented Oct 24, 2020

zowoq commented Oct 24, 2020

r-burns commented Oct 24, 2020 •

edited

Loading

zowoq commented Oct 24, 2020

r-burns commented Oct 25, 2020

zowoq commented Oct 25, 2020 •

edited

Loading

zowoq commented Oct 25, 2020

r-burns commented Oct 25, 2020

zowoq commented Oct 25, 2020 •

edited

Loading

r-burns commented Oct 25, 2020

SuperSandro2000 commented Oct 26, 2020 •

edited

Loading

SuperSandro2000 commented Oct 26, 2020

zowoq commented Oct 29, 2020

zowoq commented Nov 7, 2020

glittershark commented Nov 12, 2020

zowoq commented Nov 13, 2020

zowoq commented Nov 21, 2020

cctools-port 949.0.1 broke various builds on macOS #101330

cctools-port 949.0.1 broke various builds on macOS #101330

Comments

r-burns commented Oct 22, 2020 • edited Loading

SuperSandro2000 commented Oct 24, 2020 • edited Loading

r-burns commented Oct 24, 2020

r-burns commented Oct 24, 2020

zowoq commented Oct 24, 2020

r-burns commented Oct 24, 2020 • edited Loading

zowoq commented Oct 24, 2020

r-burns commented Oct 25, 2020

zowoq commented Oct 25, 2020 • edited Loading

zowoq commented Oct 25, 2020

r-burns commented Oct 25, 2020

zowoq commented Oct 25, 2020 • edited Loading

r-burns commented Oct 25, 2020

SuperSandro2000 commented Oct 26, 2020 • edited Loading

SuperSandro2000 commented Oct 26, 2020

zowoq commented Oct 29, 2020

zowoq commented Nov 7, 2020

glittershark commented Nov 12, 2020

zowoq commented Nov 13, 2020

zowoq commented Nov 21, 2020

r-burns commented Oct 22, 2020 •

edited

Loading

SuperSandro2000 commented Oct 24, 2020 •

edited

Loading

r-burns commented Oct 24, 2020 •

edited

Loading

zowoq commented Oct 25, 2020 •

edited

Loading

zowoq commented Oct 25, 2020 •

edited

Loading

SuperSandro2000 commented Oct 26, 2020 •

edited

Loading