Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

p4a's overtaking of dependency order in place of --no-deps is problematic and IMHO should go #1490

Closed
ghost opened this issue Nov 23, 2018 · 1 comment

Comments

@ghost
Copy link

ghost commented Nov 23, 2018

p4a's overtaking of dependency order of pure python modules in place of --no-deps is problematic and IMHO should go, even though this will break certain builds and recipes I assume.

Let me explain why I think it should go away:

  • the CythonRecipe install process is fundamentally hacky, by not using the proper setup.py but instead manually Cythonizing with 1. wrong venv, 2. wrong python, 3. wrong site-packages, which can work or can badly break things (e.g. for projects with .pxd include dependencies)
  • therefore, I attempted to make CythonRecipe unnecessary by fixing the build setup to work out of the box (see pull request Make Cython work without recipe #1483 ), allowing the regular setup.py to run instead of the limited bypassed Cythonize of p4a
  • however, the dependency graph resolving --requirements relies on the specified dependencies in the recipes to order them, and just completely jumbles up the order of non-recipe'd dependencies. However, this order is vital for projects with .pxd dependencies, and while it's probably not commonly relevant for most others, it can be and so this is just a fundamentally problematic assumption IMHO
  • but to make the dependency graph do this properly, it would need to actually somehow query the setup.py properly to find it the order of pure python modules, since there are no recipes for them to get the deps from. This is of course not an easy task and probably a bad idea due to the complexity

The obvious alternate solution is to remove --no-deps in build.py's run_pymodules_install() func. However, this has a few implications:

  1. recipes would need to be installed properly (in a way that pip recognizes they're installed) if they're dependency of any pure python module, or things will break (pip will attempt to install them twice)
  2. all dependencies of any pure python module would always be installed with no way to stop this from happening. to change this, one would need to get a custom recipe to prevent this, while previously, one could just omit these dependencies from --requirements

However, it would have the following bonuses:

  1. the flawed dependency graph will be completely avoided for pure python modules, instead of making it even more complicated and error-prone in a possibly futile attempt to fix the ordering of pure python installs
  2. it will no longer be necessary to specify dependencies-of-dependencies for pure python modules in the --requirements list, making it easier to put together all needed requirements
  3. In general, this will move p4a a good bit further towards less recipes, less custom dependency hacking, less setup.py bypassing and just letting things install as they were designed to install

I wonder if there is a better solution, so I'm interested in input on this.

@ghost
Copy link
Author

ghost commented Jun 20, 2019

This is now solved by #1625 when using a setup.py & --use-setup-py which IMHO is the most generic and cleanest way to package for complex projects anyway, I just forgot to close this!

@ghost ghost closed this as completed Jun 20, 2019
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants