update to the cctools 1009.2 and ld64 907

[wolfv]

Maybe someone can double check the patch. I initialize the IsOutputMappable instead of the previous slightly more complicated logic.

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe:

• This recipe is using a compiler, which now requires adding a build dependence on {{ stdlib("c") }} as well. Note that this rule applies to each output of the recipe using a compiler. For further details, please see META: {{ stdlib("c") }} migration conda-forge.github.io#2102.
• You're setting a constraint on the __osx virtual package directly; this should now be done by adding a build dependence on {{ stdlib("c") }}, and overriding c_stdlib_version in recipe/conda_build_config.yaml for the respective platform as necessary. For further details, please see META: {{ stdlib("c") }} migration conda-forge.github.io#2102.
• In your conda_build_config.yaml, please change the name of MACOSX_DEPLOYMENT_TARGET, to c_stdlib_version!

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

[h-vetinari]

Linux fails with

Required library libblocksruntime not installed (install libdispatch!)

Let's try with https://github.com/conda-forge/libdispatch-feedstock then...

[h-vetinari]

Seems we have a problem in the libdispatch headers

In file included from $SRC_DIR/cctools/ld64/src/ld/passes/order.cpp:30:
$PREFIX/include/dispatch/dispatch.h:42:10: fatal error: 'dispatch/unistd.h' file not found
   42 | #include <dispatch/unistd.h>
      |          ^~~~~~~~~~~~~~~~~~~
1 error generated.

That file isn't shipped by libdispatch; presumably this should just be #include <unistd.h>...? 🤔

[h-vetinari]

No idea why this isn't being picked up on osx:

$SRC_DIR/cctools/ld64/src/ld/OutputFile.cpp:7927:2: error: unknown type name 'dispatch_queue_global_t'; did you mean 'dispatch_queue_attr_t'?
        dispatch_queue_global_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0);
        ^~~~~~~~~~~~~~~~~~~~~~~
        dispatch_queue_attr_t
/Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include/dispatch/queue.h:465:1: note: 'dispatch_queue_attr_t' declared here
DISPATCH_DECL(dispatch_queue_attr);
^
/Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include/dispatch/object.h:81:58: note: expanded from macro 'DISPATCH_DECL'
                typedef struct name##_s : public dispatch_object_s {} *name##_t
                                                                       ^

We're definitely including dispatch/queue.h now...

[h-vetinari]

OK, I think upstream changed something (tpoechtrager/cctools-port@319ef50) and now expects builds of https://github.com/tpoechtrager/apple-libdispatch (which is a fork of https://github.com/apple/swift-corelibs-libdispatch) instead of the SDK...

[pkgw]

I'm hoping that #74 fixes the crashes seen in some recent Rust builds, but I kind of suspect that we'll need this PR to go in before those get solved. Is this stuck on any unsolved problems, or is it "just" that someone needs to sit down and deal with libdispatch and whatever other compiler errors that pop up?

[h-vetinari]

I kind of suspect that we'll need this PR to go in before those get solved

What makes this version more promising to fix things? There's also already yet another new upstream version.

is it "just" that someone needs to sit down and deal with libdispatch and whatever other compiler errors that pop up?

AFAIU someone needs to package the Apple specific libdispatch first.

[pkgw]

@h-vetinari I'll admit that I was just going on intuition, but indeed the update to 907 includes updates to the code that's crashing:

tpoechtrager/cctools-port@2b804a8#diff-8dc8bee92b23b56443bfe9f28288fffd27111bf6245dda491fc8c5ba4a9a8bce

[isuruf]

@conda-forge-admin rerender

[pkgw]

Oh hey, there's already a libdispatch-feedstock, based on the underlying apple/swift-corelibs-libdispatch repo. Is it too much to hope that it will work here out-of-the-box?

edit: Ahhhh OK the feedstock version is already used here on Linux, but if I'm understanding correctly, macOS needs to start using a non-OS version too.

edit 2: Looking at the diff between the tpoechtrager/apple-libdispatch and apple/swift-corelibs-libdispatch, I think the two should be compatible. There are no commits in tpoechtrager not found in apple, and the differences pretty much all look like portability fixes. Also, note that our package of libdispatch is where dispatch/unistd.h is introduced.

[pkgw]

I looked into building the external libdispatch on MacOS, and it looks like it would be quite painful. It seems to require private kernel headers that aren't part of our SDKs (specifically, firehose/tracepoint_private.h) as well as a tangle of supporting libraries that I personally find to be quite confusing.

Fortunately, with a one-liner I think that we can get the code compiling against the stock MacOS libdispatch again. My test build still errored out, but only at the testing phase.

[pkgw]

dispatch/unistd.h error is an oversight in the Conda patch — we need to distribute the new header. Should be addressed in conda-forge/libdispatch-feedstock#2.

[pkgw]

MacOS test errors are because:

Could not execute llvm-otool; llvm-otool comes with llvm 13 onwards

Basically, the this package wants to have a runtime dependency on a separate llvm-otool binary (otool-wrapper). Is that OK in this context?

[h-vetinari]

I think it's fine to depend on llvm-tools, either unversioned, or with the respective clang version we're building for.

Since llvm 18 we now also have versioned outputs (which are co-installable, but don't contain the unversioned binary); if desirable we can backport that.

[h-vetinari]

I have a variant of this PR using the newest version and one less patch: #75

[h-vetinari]

Alright, let's try this, thanks a lot @pkgw!

[jjerphan]

This PR caused problems with the linkage.

See #77.