Is awkward-cpp not linking properly in MacOS now? #3122
Labels
bug (unverified)
The problem described would be a bug, but needs to be triaged
installation
Troubles installing the package
Comments
jpivarski
added
bug (unverified)
|
@jpivarski can you inspect your installed environment to see whether |
|
If you mean run ldd on both and search through the list of symbols, I can do that. Is that what you mean? |
|
nm -gD <FOO.so> | cppfilt |
|
Of course! (If you believe me, I was thinking of Anyway, here are symbols that are in and here are symbols that are in and these are symbols in both (intersection): I'm going to submit this comment and look at it in more detail on the big (non-laptop) screen. |
|
It's unsurprising that the Python module has a lot of Python symbols that libawkward.dylib doesn't have. Some of the symbols that are in libawkward.dylib only look important, but the one that Python is complaining about, awkward::ForthMachineOf<int, int>::begin_again(std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::shared_ptr<awkward::ForthInputBuffer>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const, std::__1::shared_ptr<awkward::ForthInputBuffer>>>> const&, bool)is in fact in both, so Python really should be finding it. This is odd. |
Haha, you've been in this game a lot longer than I have. I take it as read at this point. |
|
Is |
|
I sometimes say one word when I mean another, but I don't often write it! It looks like I'm supposed to use % otool -L _ext.cpython-310-darwin.so
_ext.cpython-310-darwin.so:
@rpath/libawkward.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.100.2)Interestingly, it has the same output on libawkward.dylib (is it recursive?): % otool -L libawkward.dylib
libawkward.dylib:
@rpath/libawkward.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libc++.1.dylib (compatibility version 1.0.0, current version 1.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.100.2)In my environment that works, the output of both of the above is exactly the same as shown here. I wonder if % otool -l _ext.cpython-310-darwin.so | fgrep path
name @rpath/libawkward.dylib (offset 24)
name @rpath/libc++.1.dylib (offset 24)
path /Users/jpivarski/mambaforge/envs/test-pyarrow/lib (offset 12)
path @loader_path (offset 12)That directory, /Users/jpivarski/mambaforge/envs/test-pyarrow/lib, doesn't have any libawkward.dylib in it. In my (base) environment that works, the third path is /Users/jpivarski/mambaforge/lib, and there's no libawkward.dylib in there, either. This is very strange:
The strangeness of this makes me think that the error message shouldn't be taken literally—it's some sort of red herring. |
|
Through many repetitions, I have found the following to be absolutely reproducible. If I create an environment with the It works: If It doesn't work: So something goes wrong strictly after Using this: It looks like my Xcode commandline tools has version 15. I wonder if this is automatically updated, or if it's out of date and that's why newer conda |
|
It doesn't seem like I can update my Xcode commandline tools: % softwareupdate --list
Software Update Tool
Finding available software
No new software available.Also, it looks like Xcode 15 is the latest version from Apple. |
|
@henryiii, have you seen this elsewhere? I find that I need to constrain Incidentally, applying this restriction on |
|
Xcode and LLVM versions are close now, but they are not related. I've not seen this, though I have seen some issues with Xcode 15 producing invalid AVX instructions that break on older macOS versions. In general, I've always held that conda doesn't support local compiles, and only supports downloading binaries; PyPI packages should be used if you want to compile anything without a conda recipe, custom environment, and Conda-build. Do you have the "compilers" package? It might be that you are compiling with the system Apple-Clang but linking with LLVM's Clang? |
|
In the above tests, I didn't have the There's an issue here, but it's not an Awkward issue, so I'll be closing this soon. I think I should follow up elsewhere, though. I can get a working conda environment on MacOS if I ensure that all of the For context, I am compiling Awkward with pip (through Scikit-Build-Core, through CMake, ...). Conda only comes in because it determines which compiler that pip ends up using to do the compilation. Very likely there's a mismatch between compiler and linker and between an XCode Commandline Tools (system-wide) installation and something that conda installs within the environment. This failure mode is similar to the one I used to see in Linux when the compiler was coming from the conda environment and the linker was coming from the base system (though the work-around that I had been using for that was to include the In Linux, I rely on conda to supply a consistent set of compilers (including linker, etc.). ROOT and cppyy need to be matched with a particular compiler version/ABI, and it's easy for a version of ROOT or cppyy to not be correctly matched with a system-wide compiler, such as Apparently, what seems to me like a best practice in Linux is not a best practice in MacOS. I have a strong suspicion that the MacOS versions of the But then, the @henryiii, how should I report this? If this is "not a bug," then what's the story for the conda-forge ROOT package on MacOS? What should users like me do? |
|
(Just closing this issue, since it's not Awkward-related.) |
|
Very interesting: https://cmake.org/cmake/help/latest/variable/CMAKE_LINKER_TYPE.html
|
|
I'm not sure what this implies. Are you saying that Apple's linker changed behavior in version 15 (the version that I have) and now CMake projects like awkward-cpp need to opt-in to get correct behavior? So it could be fixed by a change in awkward-cpp's CMakeLists.txt? If so, that would be great, as it would prevent other people from running into this installation issue with Awkward Array, but it's a problem for any other developer of compiled software that might run in a conda environment with ROOT (or other package that brings in I guess if this is a fix, it needs to be widely advertised. Maybe through scientific Python because they're the ones who are most likely to be compiling in conda environments. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Version of Awkward Array
HEAD
Description and code to reproduce
I wanted to narrow in on the segfault that pyarrow 16.1.0 introduces in our tests/test_0080_flatpandas_multiindex_rows_and_columns.py test (reported in #3119 (comment)), so that I can report it to Arrow or Pandas with a minimal reproducer that does not require Awkward Array.
(There must be some step in our code that triggers the segfault; just doing
import pyarroworimport pandasdoesn't do it. I don't think it's an error in our code, but I want to use our code to find out which step in Arrow or Pandas triggers it.)To do that, I first need to compile Awkward Array in a minimal environment so that I can narrow in on the line that triggers it. Then I would install only pyarrow or pyarrow and pandas in a minimal environment, demonstrate there's a segfault, and report that to the appropriate team with the specifics of the environment.
However, I can't get that far because I can't compile Awkward in a minimal environment on MacOS. This is a minimal environment (compiler stuff and all of Awkward's required dependencies):
% mamba create --name py311-awkward python=3.11 cxx-compiler make cmake nox pytest numpy importlib_metadata packaging typing_extensions fsspecThen I do the usual dev installation in a freshly cloned Awkward repo (from
main):With this, we can't
dlopentheawkward_cpp.lib._extshared object:Using
c++filt, the offending linker object isbut I doubt that it has anything to do with this object, specifically. I suspect that the shared object can't be loaded at all, though I could be wrong.
After the Awkward installation, my environment is
awkward-cppandawkwardare the only ones "from pypi", so everything in this environment should be consistently conda-forge. Theclang,clang++,ld,make, andcmakecommands are all coming from the Conda environment, not the base system (I checked withwhich), so I'm not mixing libraries in the compilation.@ianna, do you see the same on your Mac? My base environment still works (whew!), so it must either be relying on something not listed above or some constraint in my base environment is holding back a package version that would break it. We need to know what it is that makes things work so that we don't lose that thing someday and are unable to get things working again.
The text was updated successfully, but these errors were encountered: