Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pdm install error (regarding to resolvelib) #685

Closed
stepanselyuk opened this issue Oct 15, 2021 · 16 comments
Closed

pdm install error (regarding to resolvelib) #685

stepanselyuk opened this issue Oct 15, 2021 · 16 comments
Labels
🐛 bug Something isn't working

Comments

@stepanselyuk
Copy link

  • [+] I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

pdm install -v

Actual behavior

$ pdm install -v
Traceback (most recent call last):
  File "/usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py", line 171, in _merge_into_criterion
KeyError: 'flask'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/pdm", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/pdm/core.py", line 200, in main
    return Core().main(args)
  File "/usr/local/lib/python3.9/dist-packages/pdm/core.py", line 153, in main
    raise cast(Exception, err).with_traceback(traceback)
  File "/usr/local/lib/python3.9/dist-packages/pdm/core.py", line 148, in main
    f(options.project, options)
  File "/usr/local/lib/python3.9/dist-packages/pdm/cli/commands/install.py", line 45, in handle
    actions.do_sync(
  File "/usr/local/lib/python3.9/dist-packages/pdm/cli/actions.py", line 154, in do_sync
    candidates = resolve_candidates_from_lockfile(project, requirements)
  File "/usr/local/lib/python3.9/dist-packages/pdm/cli/actions.py", line 112, in resolve_candidates_from_lockfile
    mapping, *_ = resolve(
  File "/usr/local/lib/python3.9/dist-packages/pdm/resolver/core.py", line 31, in resolve
    result = resolver.resolve(requirements, max_rounds)
  File "/usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py", line 453, in resolve
  File "/usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py", line 318, in resolve
  File "/usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py", line 173, in _merge_into_criterion
  File "/usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py", line 79, in from_requirement
TypeError: find_matches() missing 2 required positional arguments: 'requirements' and 'incompatibilities'

Expected behavior

Packages installed successfully

Environment Information

$ pdm info && pdm info --env
PDM version:        1.9.0                                    
Python Interpreter: /bin/python3 (3.9)                       
Project Root:       /home/pi/rpi-led-strip                   
Project Packages:   /home/pi/rpi-led-strip/__pypackages__/3.9
{
  "implementation_name": "cpython",
  "implementation_version": "3.9.2",
  "os_name": "posix",
  "platform_machine": "armv6l",
  "platform_release": "5.10.63+",
  "platform_system": "Linux",
  "platform_version": "#1459 Wed Oct 6 16:40:27 BST 2021",
  "python_full_version": "3.9.2",
  "platform_python_implementation": "CPython",
  "python_version": "3.9",
  "sys_platform": "linux"
}

This is a new raspbian/buster installation with upgrading to raspbian/bullseye and installation of:

sudo apt install python3 python3-pip
sudo pip3 install pdm
@stepanselyuk stepanselyuk added the 🐛 bug Something isn't working label Oct 15, 2021
@stepanselyuk
Copy link
Author

I found something similar at https://aur.archlinux.org/packages/python-pdm/#comment-810500

@stepanselyuk
Copy link
Author

stepanselyuk commented Oct 15, 2021

Somehow this worked:

$ sudo pip uninstall wheel
Found existing installation: wheel 0.37.0
Uninstalling wheel-0.37.0:
  Would remove:
    /usr/local/bin/wheel
    /usr/local/lib/python3.9/dist-packages/wheel-0.37.0.dist-info/*
    /usr/local/lib/python3.9/dist-packages/wheel/*
Proceed (y/n)? y
  Successfully uninstalled wheel-0.37.0
$ ls /usr/share/python-wheels/
appdirs-1.4.4-py2.py3-none-any.whl            pip-20.3.4-py2.py3-none-any.whl
CacheControl-0.12.6-py2.py3-none-any.whl      pkg_resources-0.0.0-py2.py3-none-any.whl
certifi-2020.6.20-py2.py3-none-any.whl        progress-1.5-py2.py3-none-any.whl
chardet-4.0.0-py2.py3-none-any.whl            pyparsing-2.4.7-py2.py3-none-any.whl
colorama-0.4.4-py2.py3-none-any.whl           requests-2.25.1-py2.py3-none-any.whl
contextlib2-0.6.0.post1-py2.py3-none-any.whl  resolvelib-0.5.4-py2.py3-none-any.whl
distlib-0.3.1-py2.py3-none-any.whl            retrying-1.3.3-py2.py3-none-any.whl
distro-1.5.0-py2.py3-none-any.whl             setuptools-44.1.1-py2.py3-none-any.whl
html5lib-1.1-py2.py3-none-any.whl             six-1.16.0-py2.py3-none-any.whl
idna-2.10-py2.py3-none-any.whl                toml-0.10.1-py2.py3-none-any.whl
ipaddr-2.2.0-py2.py3-none-any.whl             urllib3-1.26.5-py2.py3-none-any.whl
msgpack-1.0.0-py2.py3-none-any.whl            webencodings-0.5.1-py2.py3-none-any.whl
packaging-20.9-py2.py3-none-any.whl           wheel-0.34.2-py2.py3-none-any.whl
pep517-0.9.1-py2.py3-none-any.whl

Tried here, nothing worked.

$ sudo python3 -m pip install --upgrade pip
Requirement already satisfied: pip in /usr/lib/python3/dist-packages (20.3.4)
Collecting pip
  Downloading pip-21.3-py3-none-any.whl (1.7 MB)
     |████████████████████████████████| 1.7 MB 1.4 MB/s 
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.3.4
    Not uninstalling pip at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'pip'. No files were found to uninstall.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
pdm 1.9.0 requires wheel<1.0.0,>=0.36.2, but you have wheel 0.34.2 which is incompatible.
Successfully installed pip-21.3
$ sudo python3 -m pip install --upgrade wheel
Requirement already satisfied: wheel in /usr/lib/python3/dist-packages (0.34.2)
Collecting wheel
  Using cached wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel
  Attempting uninstall: wheel
    Found existing installation: wheel 0.34.2
    Uninstalling wheel-0.34.2:
      Successfully uninstalled wheel-0.34.2
Successfully installed wheel-0.37.0

Tried here - pdm install -v worked!

Meanwhile content of /usr/share/python-wheels does not seem to be changed:

/usr/share/python-wheels $ ls
appdirs-1.4.4-py2.py3-none-any.whl            pip-20.3.4-py2.py3-none-any.whl
CacheControl-0.12.6-py2.py3-none-any.whl      pkg_resources-0.0.0-py2.py3-none-any.whl
certifi-2020.6.20-py2.py3-none-any.whl        progress-1.5-py2.py3-none-any.whl
chardet-4.0.0-py2.py3-none-any.whl            pyparsing-2.4.7-py2.py3-none-any.whl
colorama-0.4.4-py2.py3-none-any.whl           requests-2.25.1-py2.py3-none-any.whl
contextlib2-0.6.0.post1-py2.py3-none-any.whl  resolvelib-0.5.4-py2.py3-none-any.whl
distlib-0.3.1-py2.py3-none-any.whl            retrying-1.3.3-py2.py3-none-any.whl
distro-1.5.0-py2.py3-none-any.whl             setuptools-44.1.1-py2.py3-none-any.whl
html5lib-1.1-py2.py3-none-any.whl             six-1.16.0-py2.py3-none-any.whl
idna-2.10-py2.py3-none-any.whl                toml-0.10.1-py2.py3-none-any.whl
ipaddr-2.2.0-py2.py3-none-any.whl             urllib3-1.26.5-py2.py3-none-any.whl
msgpack-1.0.0-py2.py3-none-any.whl            webencodings-0.5.1-py2.py3-none-any.whl
packaging-20.9-py2.py3-none-any.whl           wheel-0.34.2-py2.py3-none-any.whl
pep517-0.9.1-py2.py3-none-any.whl

@frostming
Copy link
Collaborator

Try to upgrade the version of resolvelib in /usr/share/python-wheels

@frostming
Copy link
Collaborator

I don't know how the distro works but pdm pinned "resolvelib>=0.7.0,<0.8.0". So it is better not to install pdm in the global environment.

@stepanselyuk
Copy link
Author

@frostming yes, but for example controlling a led strip requires sudo, and so pdm should be in that environment or I will need to dance around with sudo / PATH / etc. It seems to upgrade pip and wheels helped in this case.

@henri-hulski
Copy link

henri-hulski commented Nov 4, 2021

Why is this closed? Is there a fix?
I have exactly the same Problem.
I'm on Debian 11 (Bullseye), which is the newest stable.
The content from /usr/share/python-wheels is provided by the Debian package python-pip-whl (20.3.4-4) which is a dependency of python3.9-venv (3.9.2-1), python3-virtualvenv (20.4.0+ds-2) and python3-pip (20.3.4-4). python3.9-venv is a dependency of python3.9 so no way to uninstall it.
As I understand the only way to upgrade /usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl is to install an updated version of python-pip-whl, which is out of our scope.
So I think PDM should ship with it's own version of resolvelib included. Otherwise it will fail on all current Linux distributions which derived from Debian and didn't address this issue.

@frostming
Copy link
Collaborator

frostming commented Nov 4, 2021

@henri-hulski you should never mess with the system dependencies, but instead, install PDM in an isolated env. Look at the installation guide, we never recommend installing PDM into the global site-packages.

Plus, it is unfair to ask for every package that has conflicts with pip's dependencies to bundle all deps with it, that is not how Python packaging works.

@henri-hulski
Copy link

henri-hulski commented Nov 4, 2021

@frostming I didn't. Actually I followed exactly the instructions and used the install-pdm.py installer.

So it seems that the installer does't create actually a totally isolated env.

@henri-hulski
Copy link

My environment:

$ pdm info && pdm info --env
PDM version:        1.10.0
Python Interpreter: /usr/bin/python3 (3.9)
Project Root:       ~/web/myproject/myproject-server
Project Packages:   ~/web/myproject/myproject-server/__pypackages__/3.9
{
  "implementation_name": "cpython",
  "implementation_version": "3.9.2",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.14.0-0.bpo.2-amd64",
  "platform_system": "Linux",
  "platform_version": "#1 SMP Debian 5.14.9-2~bpo11+1 (2021-10-10)",
  "python_full_version": "3.9.2",
  "platform_python_implementation": "CPython",
  "python_version": "3.9",
  "sys_platform": "linux"
}

So the Python interpreter in use is the global one: /usr/bin/python3.
The same with the environment of @stepanselyuk (here /bin/python3).
So you cannot really say that it's isolated.

@frostming
Copy link
Collaborator

frostming commented Nov 4, 2021

no, pdm info shows the project environment, not the environment where pdm is

@henri-hulski
Copy link

Aa ok. is there a way to check which environment pdm is using?

@pawamoy
Copy link
Contributor

pawamoy commented Nov 4, 2021

head -n1 $(type pdm | grep -o '/.*') to show the interpreter used by PDM?

@henri-hulski
Copy link

henri-hulski commented Nov 4, 2021

This is my error, when trying to make an install:

$ pdm install --prod -vvv
Lock file does not exist or is incompatible, trying to generate one...
======== Start resolving requirements ========
  pytest-cov
  coverage[toml]
  more.cerberus
  more.jwtauth
  more.pony
  more.emit
  more.cors
  argon2-cffi
  gunicorn
  itsdangerous
  yagmail[all]
  keyrings.alt
  email-validator
  psycopg2
  pyyaml
  flake8
  black
  pytest>=2.9.1
  pytest-env
  pytest-localserver
  WebTest>=2.0.14
  python>=3.9
  Adding requirement pytest-cov
Traceback (most recent call last):
  File "/usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py", line 171, in _merge_into_criterion
KeyError: 'pytest-cov'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/.local/bin/pdm", line 8, in <module>
    sys.exit(main())
  File "~/.local/share/pdm/venv/lib/python3.9/site-packages/pdm/core.py", line 200, in main
    return Core().main(args)
  File "~/.local/share/pdm/venv/lib/python3.9/site-packages/pdm/core.py", line 153, in main
    raise cast(Exception, err).with_traceback(traceback)
  File "~.local/share/pdm/venv/lib/python3.9/site-packages/pdm/core.py", line 148, in main
    f(options.project, options)
  File "~/.local/share/pdm/venv/lib/python3.9/site-packages/pdm/cli/commands/install.py", line 39, in handle
    actions.do_lock(project, strategy="all", dry_run=options.dry_run)
  File "~/.local/share/pdm/venv/lib/python3.9/site-packages/pdm/cli/actions.py", line 68, in do_lock
    mapping, dependencies = resolve(
  File "~/.local/share/pdm/venv/lib/python3.9/site-packages/pdm/resolver/core.py", line 31, in resolve
    result = resolver.resolve(requirements, max_rounds)
  File "/usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py", line 453, in resolve
  File "/usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py", line 318, in resolve
  File "/usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py", line 173, in _merge_into_criterion
  File "/usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py", line 79, in from_requirement
TypeError: find_matches() missing 2 required positional arguments: 'requirements' and 'incompatibilities'

So it seems that ~/.local/share/pdm/venv/lib/python3.9/site-packages/pdm/resolver/core.py is using the global /usr/share/python-wheels/resolvelib-0.5.4-py2.py3-none-any.whl/resolvelib/resolvers.py.

@henri-hulski
Copy link

henri-hulski commented Nov 4, 2021

So it seems that the Problem comes from the import of Resolver.
https://github.com/pdm-project/pdm/blob/main/pdm/resolver/core.py#L9-L10

if TYPE_CHECKING:
    from resolvelib.resolvers import Resolver

@frostming
Copy link
Collaborator

frostming commented Nov 4, 2021

It turns out the redistributed pip has a special debundling process that adds the wheels path in front of sys.path.

As a workaround, you can re-install the pip in the venv by:

~/.local/share/pdm/venv/bin/pip install -I pip

@iharh
Copy link

iharh commented Nov 5, 2021

Thank you very much. The proposed workaround (re-install the pip in the venv) has just helped me in the same situation, as described.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants