Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-41100: ctypes: check _dyld_shared_cache_contains_path in find_library #21241

Closed

Conversation

lawrence-danna-apple
Copy link
Contributor

@lawrence-danna-apple lawrence-danna-apple commented Jun 30, 2020

System libraries in Mac OS 11 may be present only in the shared cache,
with the actual mach-o file not present in the corresponding location
on the filesystem. ctypes.util.find_library should check the
shared cache in order to behave consistently across Mac OS 10.15
and earlier and Mac OS 11.0 and later.

https://bugs.python.org/issue41100

@ronaldoussoren
Copy link
Contributor

Is it necessary to add _dyld_shared_cache_contains_path, or is safe to assume libraries in /usr/lib and /lib are in the cache?

In PyObjC similar code to find_library assumes that a library is in the shared cache if (a) the library path starts with "/usr/lib" or "/lib" and (b) there is a symbolic link (such as /usr/lib/libSystem.dylib).

I'm not too happy about using a private symbol, that tends to be fragile.

@lawrence-danna-apple
Copy link
Contributor Author

lawrence-danna-apple commented Jun 30, 2020

"I'm not too happy about using a private symbol, that tends to be fragile"

I just spoke to dyld engineers, _dyld_shared_cache_contains_path will be made a public API in Mac OS 11 Seed 3

I was thinking we'd add it as a private symbol now, and then update it to including dyld.h when Seed 3 is out.

@lawrence-danna-apple
Copy link
Contributor Author

"Is it necessary to add _dyld_shared_cache_contains_path, or is safe to assume libraries in /usr/lib and /lib are in the cache?"

I think it is necessary, because if we just assumed any name under /lib was in the cache, then it would think literally anything you ask for is in the cache. find_library("FOO") would return /lib/libFOO.dylib, which is not what we want.

System libraries in Mac OS 11 may be present only in the shared cache,
with the actual mach-o file not present in the corresponding location
on the filesystem.   ctypes.util.find_library should check the
shared cache in order to behave consistently across Mac OS 10.15
and earlier and Mac OS 11.0 and later.
@@ -64,6 +64,10 @@
#include "ctypes_dlfcn.h"
#endif

#ifdef __APPLE__
extern bool _dyld_shared_cache_contains_path(const char* path) __attribute__((weak_import));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be replaced with #include <mach-o/dyld.h> after Seed 3

Copy link

@jeremyhu jeremyhu Jul 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please wait for official API here? or at least have another PR to clean this up which we can merge later?

Copy link

@kainjow kainjow Jul 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seed 3 is out today 🎉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like it got pushed out to seed 4

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is in Xcode 12 beta 4, only enough with the same name but with documentation in a header file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, updated the pull request to include mach-o/dyld.h instead of declaring a weak import

samschott pushed a commit to samschott/maestral-cocoa that referenced this pull request Jul 24, 2020
... and wait for Python fix in python/cpython#21241
@ned-deily
Copy link
Member

Thanks for the PR. It has been included in and superseded by GH-22855,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants