Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About wheel_repair.py for Windows #3

Open
myselfhimself opened this issue Nov 7, 2020 · 4 comments
Open

About wheel_repair.py for Windows #3

myselfhimself opened this issue Nov 7, 2020 · 4 comments

Comments

@myselfhimself
Copy link

Hello I have not tried your CLI wheel repair script yet, but it is fresher and slightly more advanced than machomangler which it wraps and leverages.
Have you thought about seperating it into a pypi.org package, maybe with a more windows-related name ?

Related cibuildwheel Windows wheel repair issue: pypa/cibuildwheel#459

@vinayak-mehta
Copy link
Owner

vinayak-mehta commented Nov 8, 2020

Have you thought about seperating it into a pypi.org package, maybe with a more windows-related name ?

I thought about it but didn't do it because the script has some limitations right now (which doesn't affect my very simple extension):

  • it assumes that there’s just one PYD file in the wheel which might not be the case for complex projects with multiple PYDs
  • it doesn’t look for DLLs in system directories, only the vcpkg installation directory (as I wasn’t sure what all directories to look into)

I would love to hear about your use-case before thinking about packaging this. Do you have multiple PYDs in the wheel you're trying to build, are you using vcpkg, and what are the build systems behind C/C++ code in your extension, etc. We can also get on a call if you're available.

@myselfhimself
Copy link
Author

Hello,
I think that those limitations are not big drawbacks and do not prevent from releasing your wheel repair project as a pypi.org package with its own Github/* repository, as long as you indeed mention those 2 known issues.
Also publishing it in such a separate way would attract other people to come and make pull requests.
Imperfection is sometimes perfect for now.

@zooba, you seem literate in this topic, would you have thoughts to share, if time allows it?

@vinayak-mehta
My use case is for gmic-py, a Python binding around a C++ library named libgmic (or G'MIC)
The project just generates a single .dll (or .so depending on the OS) which is a recompilation of libgmic (so, static in someway), with a light C Python API layer (in gmicpy.cpp), and depends dynamically on libpng, libjpeg, libtiff, zlib, libfftw, lipomb (OpenMP support), libcurl, not more shared libraries I think.
Its only Python dependency is wurlitzer, which is a pure python package, allowing for automatic stdout/stderr redirections.
The project has no init.py file or any pure-Python files for the end user, apart from pytests and setup.py.

So since I want to avoid pip install users to install all the dynamic libraries on Windows.. I am looking for a 1 command trick, hence .your wheel repair command. MSYS2 is the currently targeted environment, because our project leader liked compiling libgmic on MSYS2 which makes feel like on Linux on top of Windows... so I am continuing with MSYS2. Note that in an MSYS2 environment, .dll dependencies introspection can be done at least supercially with the unix-like builtin "ldd" command.

For now, this not yet cibuildwheel-enabled windows Github Action job log shows 2 artifacts which are :

  1. gmic-py's compiled .dll file for direction import gmic
  2. gmic-py's .whl file containing that same .dll file, but I fail to include other dlls for now at or before wheel-build-time, because I do understand which pieces of code and toggles to enable in setup.py / Manifest.in... If some tool would repair my build a wheel post-build time, without my need to fiddle with those manifest/setup.py package params, it would also be fine.

myselfhimself added a commit to myselfhimself/pdftopng that referenced this issue Nov 9, 2020
@zooba
Copy link

zooba commented Nov 12, 2020

Not sure what thoughts you want me to have...

I dislike this entire approach, regardless of operating system, and would rather people used stable/forward-compatible APIs. I don't want to have to debug the potential resource and name conflicts, and it bothers me that we take this approach to try and automatically fix issues that either need to be fixed by having conflicting packages communicate with each other, or by establishing package demands in a way that distributors and users can resolve it.

Those probably weren't what you were hoping for :) Might have to be more specific

@myselfhimself
Copy link
Author

myselfhimself commented Nov 13, 2020

I must agree, if I understand well, that the machomangling step seems perfectionist to me... What I wished in a repair tool for Windows was just: 1) inspect the needed third-party .dll files, 2) put them in the wheel archive

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants