Major performance regression with lief 0.14 on osx

[h-vetinari]

In the GCP bindings feedstock, the osx builds used to be around 4-5h runtime with lief 0.12. With the first PR after the release of 0.14, the osx builds consistently ran into the 6h timeout.

Looking closer, the lief processing with 0.14 took about 1:20h (CI run)

2024-04-18T17:33:19.8798900Z WARNING: Partially parsed some '.tbd' files in sysroot /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk, pretending .tbds are their install-names
2024-04-18T17:33:19.8829080Z Adding support to 'conda-build' for parsing these in 'liefldd.py' would be easy and useful:
2024-04-18T17:33:19.8856060Z ['System/Library/PrivateFrameworks/AVConference.framework/Frameworks/LegacyHandle.framework/Versions/A/LegacyHandle.tbd', 'System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/CarbonSound.framework/Versions/A/CarbonSound.tbd'] ...
2024-04-18T18:54:59.0836790Z    INFO: sysroot: '/Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/' files: '['usr/share/man/mann/zero.n', 'usr/share/man/mann/yencode.n', 'usr/share/man/mann/yaml.n', 'usr/share/man/mann/xsxp.n']'
2024-04-18T18:55:15.9321690Z    INFO (libgoogle-cloud-all,lib/libgoogle_cloud_cpp_baremetalsolution.2.23.0.dylib): Needed DSO lib/libgoogle_cloud_cpp_grpc_utils.2.dylib found in conda-forge/osx-64::libgoogle-cloud==2.23.0=h651e89d_0
2024-04-18T18:55:15.9330620Z    INFO (libgoogle-cloud-all,lib/libgoogle_cloud_cpp_baremetalsolution.2.23.0.dylib): Needed DSO lib/libgoogle_cloud_cpp_common.2.dylib found in conda-forge/osx-64::libgoogle-cloud==2.23.0=h651e89d_0

whereas with 0.12, it takes ~20 seconds (CI run):

2024-04-19T03:33:06.3979460Z WARNING: Partially parsed some '.tbd' files in sysroot /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk, pretending .tbds are their install-names
2024-04-19T03:33:06.3980840Z Adding support to 'conda-build' for parsing these in 'liefldd.py' would be easy and useful:
2024-04-19T03:33:06.3981540Z ['usr/lib/libbz2.tbd', 'System/Library/Frameworks/Collaboration.framework/Versions/A/Collaboration.tbd'] ...
2024-04-19T03:33:26.2088490Z    INFO: sysroot: '/Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/' files: '['usr/share/man/mann/zero.n', 'usr/share/man/mann/yencode.n', 'usr/share/man/mann/yaml.n', 'usr/share/man/mann/xsxp.n']'
2024-04-19T03:33:26.2182850Z    INFO (libgoogle-cloud-all,lib/libgoogle_cloud_cpp_essentialcontacts.2.23.0.dylib): Needed DSO lib/libgoogle_cloud_cpp_grpc_utils.2.dylib found in conda-forge/osx-64::libgoogle-cloud==2.23.0=h651e89d_0
2024-04-19T03:33:26.2193590Z    INFO (libgoogle-cloud-all,lib/libgoogle_cloud_cpp_essentialcontacts.2.23.0.dylib): Needed DSO lib/libgoogle_cloud_cpp_common.2.dylib found in conda-forge/osx-64::libgoogle-cloud==2.23.0=h651e89d_0

[h-vetinari]

CC @conda-forge/core, if you see big jumps in osx CI runtime on a feedstock, it could be the new lief version.

[wolfv]

I know this is maybe not exactly useful information, but in rattler-build we're using the goblin Rust crate for detecting what packages are linked and so on.

If the conda-build maintainers would be interested, we could build some Python bindings and share some code, and potentially rip out the whole LIEF / otool / ldd business in conda-forge. There is a lot of (hard to read) code in there.

And for us it would be exciting to share this code :)

[h-vetinari]

The performance regression is also very visible in for the clang-activation builds, where we went from ~20 seconds for 18.1.2

2024-03-22T03:58:04.8819220Z Skipping -add_rpath, file already has LC_RPATH set
2024-03-22T03:58:05.1460280Z Skipping -add_rpath, file already has LC_RPATH set
2024-03-22T03:58:23.9979080Z    INFO: sysroot: '/Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/' files: '['usr/share/man/mann/zero.n', 'usr/share/man/mann/yencode.n', 'usr/share/man/mann/yaml.n', 'usr/share/man/mann/xsxp.n']'
2024-03-22T03:58:24.0476660Z    INFO (clang_bootstrap_osx-64,bin/x86_64-apple-darwin13.4.0-codesign_allocate): Needed DSO /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib/libSystem.B.dylib found in $SYSROOT
2024-03-22T03:58:24.0580930Z WARNING (clang_bootstrap_osx-64,bin/llvm-jitlink): Needed DSO lib/libLLVM.18.1.dylib found in ['conda-forge/osx-64::libllvm18==18.1.2=hbcf5fad_0'] (and also in this package)
2024-03-22T03:58:24.0582480Z WARNING (clang_bootstrap_osx-64,bin/llvm-jitlink): .. but ['conda-forge/osx-64::libllvm18==18.1.2=hbcf5fad_0'] not in reqs/run, (i.e. it is overlinking) (likely) or a missing dependency (less likely)

to 30 minutes for 18.1.3

2024-04-08T23:08:19.3583750Z Skipping -add_rpath, file already has LC_RPATH set
2024-04-08T23:08:19.8354910Z Skipping -add_rpath, file already has LC_RPATH set
2024-04-08T23:38:58.9503870Z    INFO: sysroot: '/Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/' files: '['usr/share/man/mann/zero.n', 'usr/share/man/mann/yencode.n', 'usr/share/man/mann/yaml.n', 'usr/share/man/mann/xsxp.n']'
2024-04-08T23:38:59.3330680Z    INFO (clang_bootstrap_osx-64,libexec/as/ppc64/as): Needed DSO /Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib/libSystem.B.dylib found in $SYSROOT
2024-04-08T23:38:59.9828890Z WARNING (clang_bootstrap_osx-64,bin/llvm-lto): Needed DSO lib/libLLVM.18.1.dylib found in ['conda-forge/osx-64::libllvm18==18.1.3=hbcf5fad_0'] (and also in this package)
2024-04-08T23:38:59.9830020Z WARNING (clang_bootstrap_osx-64,bin/llvm-lto): .. but ['conda-forge/osx-64::libllvm18==18.1.3=hbcf5fad_0'] not in reqs/run, (i.e. it is overlinking) (likely) or a missing dependency (less likely)

That's a regression by almost a factor ~100! 😱

[mbargull]

This has been a downstream issue introduced by yours truly and fixed in conda-build=24.5.1 (which should be available in a couple of hours on conda-forge) via conda/conda-build#5348 .
Please let me know in case you still encounter such regressions with the newer conda-build.

[h-vetinari]

Thanks for the fix! 🙏

[h-vetinari]

(and the work on the lief updates in the first place!)

[h-vetinari]

I looked at the build times a bit more closely again today, and even with conda-build 24.5.1, the clang-activation feedstock still takes ~25min for the link check (compared to ~20sec with older lief):

2024-07-29T07:19:51.0180240Z ['System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/NavigationServices.framework/NavigationServices.tbd', 'System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics.tbd']...
2024-07-29T07:45:20.0253070Z    INFO: sysroot: '/Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/' files: '['usr/share/man/mann/zero.n', 'usr/share/man/mann/yencode.n', 'usr/share/man/mann/yaml.n', 'usr/share/man/mann/xsxp.n']'

So I think this issue has to be reopened.

[h-vetinari]

FWIW, the 25min were on a slow agent, but even the very fastest osx build I found takes ~17min, which is still orders of magnitude worse than ~20sec.

2024-07-29T07:18:35.1611990Z ['System/iOSSupport/System/Library/Frameworks/CarPlay.framework/CarPlay.tbd', 'usr/lib/libf77lapack.tbd']...
2024-07-29T07:35:22.3511700Z    INFO: sysroot: '/Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/' files: '['usr/share/man/mann/zero.n', 'usr/share/man/mann/yencode.n', 'usr/share/man/mann/yaml.n', 'usr/share/man/mann/xsxp.n']'

[wolfv]

Would you be interested in us factoring out the goblin parts of rattler-build to make them available to conda-build as an alternative to lief?

[h-vetinari]

A recent change in the way smithy pins the python version (vs. being able to override it in remote_ci_setup) caused me to switch back to lief 0.14 for google-cloud-cpp; Compared to 30 seconds for the link check on osx with 0.12 before, it now takes >125min, so roughly a 250-fold regression.