pdm with python2 adding mypy error needs better error message #338

linw1995 · 2021-03-24T10:02:16Z

I have searched the issue tracker and believe that this is not a duplicate.

The project needs Python 3.5 or later to use mypy. Of course, it adds mypy failure but pdm needs to print a better error message.

Steps to reproduce

initialize a new project with python2.7
run pdm add mypy

Actual behavior

adding package command run failure

Adding packages to default dependencies: mypy
Resolving: mypy 0.720
Resolving: mypy-extensions 0.4.3
Resolving: typing-extensions 3.7.4.3
Resolving: typing 3.7.4.1
Resolving: typed-ast 1.4.2
Finish resolving
Extracting package metadata
🔒 Lock successful
Changes are written to pdm.lock.
Changes are written to pyproject.toml.
Synchronizing working set with lock file: 5 to add, 0 to update, 0 to remove

Install mypy-extensions 0.4.3 successful
Install typing 3.7.4.1 successful
Install typing-extensions 3.7.4.3 successful
Install typed-ast 1.4.2 failed
Install mypy 0.720 failed
  Retry failed jobs
Install typed-ast 1.4.2 failed
Install mypy 0.720 failed

ERRORS:
add typed-ast failed:
Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/synchronizers.py", line 147, in install_candidate
    installer.install(can)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/installers.py", line 48, in install
    self.install_wheel(candidate.wheel)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/installers.py", line 54, in install_wheel
    wheel.install(paths, maker)
AttributeError: 'NoneType' object has no attribute 'install'
add mypy failed:
Traceback (most recent call last):
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/builders.py", line 90, in log_subprocessor
    subprocess.check_call(
  File "/usr/local/Cellar/[email protected]/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Users/linw1995/.pyenv/versions/2.7.18/bin/python2', '/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pep517/in_process/_in_process.py', 'get_requires_for_build_wheel', '/var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/tmp49zo5fw9']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/candidates.py", line 153, in get_metadata
    built = self.environment.build(ireq, self.hashes, allow_all_wheels)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/environment.py", line 313, in build
    return builder.build_wheel(output_dir)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/builders.py", line 253, in build_wheel
    requires = self._hook.get_requires_for_build_wheel()
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pep517/wrappers.py", line 168, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pep517/wrappers.py", line 265, in _call_hook
    self._subprocess_runner(
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/builders.py", line 161, in subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/builders.py", line 98, in log_subprocessor
    raise BuildError(f"Call command {cmd} return non-zero status.")
pdm.exceptions.BuildError: Call command ['/Users/linw1995/.pyenv/versions/2.7.18/bin/python2', '/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pep517/in_process/_in_process.py', 'get_requires_for_build_wheel', '/var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/tmp49zo5fw9'] return non-zero status.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/synchronizers.py", line 147, in install_candidate
    installer.install(can)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/installers.py", line 44, in install
    candidate.get_metadata(allow_all_wheels=False)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/candidates.py", line 156, in get_metadata
    meta_dict = SetupReader.read_from_directory(ireq.unpacked_source_directory)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/readers.py", line 35, in read_from_directory
    new_result = getattr(cls(), "read_{}".format(filename.replace(".", "_")))(
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/readers.py", line 80, in read_setup_py
    setup_call, body = self._find_setup_call(body)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/readers.py", line 182, in _find_setup_call
    and func.value.id == "setuptools"
AttributeError: 'Attribute' object has no attribute 'id'

See /var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/pdm-install-mdqcegbs.log for detailed debug log.
[InstallationError]: Some package operations are not complete yet
Add '-v' to see the detailed traceback

add with '-v'

Adding packages to default dependencies: mypy
======== Start resolving requirements ========
	mypy
	Adding requirement mypy
	Adding requirement typed-ast<1.5.0,>=1.4.0(from mypy 0.720)
	Adding requirement typing-extensions>=3.7.4(from mypy 0.720)
	Adding requirement mypy-extensions<0.5.0,>=0.4.0(from mypy 0.720)
Resolving: mypy 0.720
	New pin: mypy 0.720
======== Ending round 0 ========
	Adding requirement typing>=3.5.3; python_version < "3.5"(from mypy-extensions 0.4.3)
Resolving: mypy-extensions 0.4.3
	New pin: mypy-extensions 0.4.3
======== Ending round 1 ========
	Adding requirement typing>=3.7.4; python_version < "3.5"(from typing-extensions 3.7.4.3)
Resolving: typing-extensions 3.7.4.3
	New pin: typing-extensions 3.7.4.3
======== Ending round 2 ========
Resolving: typing 3.7.4.1
	New pin: typing 3.7.4.1
======== Ending round 3 ========
Resolving: typed-ast 1.4.2
	New pin: typed-ast 1.4.2
======== Ending round 4 ========
Finish resolving
======== Resolution Result ========
Stable pins:
               mypy 0.720
    mypy-extensions 0.4.3
  typing-extensions 3.7.4.3
             typing 3.7.4.1
          typed-ast 1.4.2
Extracting package metadata
🔒 Lock successful
Changes are written to pdm.lock.
Changes are written to pyproject.toml.
Synchronizing working set with lock file: 5 to add, 0 to update, 0 to remove

Install mypy-extensions 0.4.3 successful
Install typing 3.7.4.1 successful
Preparing isolated env for PEP 517 build...
Install typing-extensions 3.7.4.3 successful
Preparing isolated env for PEP 517 build...
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Requirement already satisfied: setuptools>=40.8.0 in /usr/local/lib/python2.7/site-packages (from -r /var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/pdm-build-reqs-cwo97wkq.txt (line 1)) (41.4.0)
Requirement already satisfied: wheel in /usr/local/lib/python2.7/site-packages (from -r /var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/pdm-build-reqs-cwo97wkq.txt (line 2)) (0.33.6)
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Requirement already satisfied: setuptools>=40.8.0 in /usr/local/lib/python2.7/site-packages (from -r /var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/pdm-build-reqs-4yc002oo.txt (line 1)) (41.4.0)
Requirement already satisfied: wheel in /usr/local/lib/python2.7/site-packages (from -r /var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/pdm-build-reqs-4yc002oo.txt (line 2)) (0.33.6)
Error: typed_ast only runs on Python 3.3 and above.
Failed to build package, try parsing project files.
Install typed-ast 1.4.2 failed
ERROR: You need Python 3.5 or later to use mypy.
Failed to build package, try parsing project files.
Install mypy 0.720 failed
  Retry failed jobs
Install typed-ast 1.4.2 failed
Preparing isolated env for PEP 517 build...
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Requirement already satisfied: setuptools>=40.8.0 in /usr/local/lib/python2.7/site-packages (from -r /var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/pdm-build-reqs-m_fhf4lj.txt (line 1)) (41.4.0)
Requirement already satisfied: wheel in /usr/local/lib/python2.7/site-packages (from -r /var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/pdm-build-reqs-m_fhf4lj.txt (line 2)) (0.33.6)
ERROR: You need Python 3.5 or later to use mypy.
Failed to build package, try parsing project files.
Install mypy 0.720 failed

ERRORS:
add typed-ast failed:
Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/synchronizers.py", line 147, in install_candidate
    installer.install(can)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/installers.py", line 48, in install
    self.install_wheel(candidate.wheel)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/installers.py", line 54, in install_wheel
    wheel.install(paths, maker)
AttributeError: 'NoneType' object has no attribute 'install'
add mypy failed:
Traceback (most recent call last):
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/builders.py", line 90, in log_subprocessor
    subprocess.check_call(
  File "/usr/local/Cellar/[email protected]/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Users/linw1995/.pyenv/shims/python', '/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pep517/in_process/_in_process.py', 'get_requires_for_build_wheel', '/var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/tmp5276ylln']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/candidates.py", line 153, in get_metadata
    built = self.environment.build(ireq, self.hashes, allow_all_wheels)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/environment.py", line 313, in build
    return builder.build_wheel(output_dir)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/builders.py", line 253, in build_wheel
    requires = self._hook.get_requires_for_build_wheel()
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pep517/wrappers.py", line 168, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pep517/wrappers.py", line 265, in _call_hook
    self._subprocess_runner(
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/builders.py", line 161, in subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/builders.py", line 98, in log_subprocessor
    raise BuildError(f"Call command {cmd} return non-zero status.")
pdm.exceptions.BuildError: Call command ['/Users/linw1995/.pyenv/shims/python', '/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pep517/in_process/_in_process.py', 'get_requires_for_build_wheel', '/var/folders/h5/9g_q36ss4zn092x45_k1gx7w0000gn/T/tmp5276ylln'] return non-zero status.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.9.1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/synchronizers.py", line 147, in install_candidate
    installer.install(can)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/installers.py", line 44, in install
    candidate.get_metadata(allow_all_wheels=False)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/candidates.py", line 156, in get_metadata
    meta_dict = SetupReader.read_from_directory(ireq.unpacked_source_directory)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/readers.py", line 35, in read_from_directory
    new_result = getattr(cls(), "read_{}".format(filename.replace(".", "_")))(
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/readers.py", line 80, in read_setup_py
    setup_call, body = self._find_setup_call(body)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/models/readers.py", line 182, in _find_setup_call
    and func.value.id == "setuptools"
AttributeError: 'Attribute' object has no attribute 'id'

Traceback (most recent call last):
  File "/Users/linw1995/.local/bin/pdm", line 8, in <module>
    sys.exit(main())
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/core.py", line 171, in main
    return Core().main(args)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/core.py", line 133, in main
    raise err.with_traceback(traceback)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/core.py", line 128, in main
    f(options.project, options)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/cli/commands/add.py", line 35, in handle
    actions.do_add(
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/cli/actions.py", line 182, in do_add
    do_sync(
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/cli/actions.py", line 129, in do_sync
    handler.synchronize(clean=clean, dry_run=dry_run)
  File "/Users/linw1995/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/synchronizers.py", line 332, in synchronize
    raise InstallationError("Some package operations are not complete yet")
pdm.exceptions.InstallationError: Some package operations are not complete yet

Expected behavior

Adding package command should run failure, but with a better error message.

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
PDM version:        1.4.3
Python Interpreter: /Users/linw1995/.pyenv/shims/python (2.7.17)
Project Root:       /private/tmp/demo
{
  "os_name": "posix",
  "python_version": "2.7",
  "platform_system": "Darwin",
  "platform_release": "20.3.0",
  "implementation_version": "0",
  "python_full_version": "2.7.17",
  "implementation_name": "",
  "platform_python_implementaiton": "CPython",
  "platform_machine": "x86_64",
  "sys_platform": "darwin",
  "platform_version": "Darwin Kernel Version 20.3.0: Thu Jan 21 00:07:06 PST 2021; root:xnu-7195.81.3~1/RELEASE_X86_64"
}

The text was updated successfully, but these errors were encountered:

frostming · 2021-03-24T14:37:54Z

Thanks for catching this. It is more than an output format issue, some bugs lie there. mypy shouldn't even have been resolved successfully on Python 2.7.

frostming · 2021-03-24T15:21:49Z

It turns out mypy uploaded a malformed wheel mypy 0.720 that DOESN'T contain python-requires constraint and got pick up by the resolver. Based on some assumption pdm thought the candidate should be compatible with python 2.7 while it actually needs Python 3.6+.

Based on the above analysis, there is little pdm can do on this rarely happening case. Users can anyway read the error from -v output and the line ERROR: You need Python 3.5 or later to use mypy. can tell why it failed.

I also submitted a PR to make pdm quit early when the wheel is failed to build on install time, avoiding some mysterious error messages like AttributeError: 'Attribute' object has no attribute 'id'

frostming · 2021-03-24T15:26:44Z

You can try the master branch to see how the error message changes, not much though.

linw1995 · 2021-03-24T15:45:42Z

Good work. Some error messages go away now.

You can try the master branch to see how the error message changes, not much though.

I install the latest change on master branch with pipx. It seems unable to install ……

pipx install git+https://github.com/pdm-project/pdm.git

It will install failure and will tell me that the git-lfs command is missing. IMHO, the document needs to explain how to install the latest version of pdm, and avoid other ones to find out then be confused.

frostming · 2021-03-25T01:54:43Z

I can install successfully on newly created ubuntu docker image, without git lfs installed
Installing a git repo doesn't need git lfs at all, can you git clone the repo successfully?

linw1995 · 2021-03-25T02:24:38Z

I remove the git-lfs command and execute git clone. fail again and output pasted below.

❯ git clone https://github.com/pdm-project/pdm.git
Cloning into 'pdm'...
remote: Enumerating objects: 580, done.
remote: Counting objects: 100% (580/580), done.
remote: Compressing objects: 100% (237/237), done.
remote: Total 8017 (delta 326), reused 534 (delta 304), pack-reused 7437
Receiving objects: 100% (8017/8017), 5.19 MiB | 118.00 KiB/s, done.
Resolving deltas: 100% (5032/5032), done.
git-lfs filter-process: git-lfs: command not found
fatal: the remote end hung up unexpectedly
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

git which version is 2.25.1 from ubuntu clones success without lfs.
on my macOS laptop, git which version is 2.29.2 clones success but checkout fail without lfs

frostming · 2021-03-25T02:26:46Z

For both cases, can you check whether the files under tests/fixtures/artifacts/ are downloaded?

linw1995 · 2021-03-25T02:51:33Z

git version 2.25.1 downloaded
2.29.2 has no files in this directory

I upgrade the git to the latest stable version 2.31.0, it clones with the same error… both can be fixed by installing the git-lfs.

linw1995 · 2021-03-25T02:57:54Z

The best way to install the latest change of the master branch is by installing via archive URL. (:」∠)_

https://github.com/pdm-project/pdm/archive/1eec71944736320b3025efea608bf1d3b9fa3081.zip

linw1995 added the 🐛 bug Something isn't working label Mar 24, 2021

linw1995 changed the title ~~pdm with python2 add mypy error~~ pdm with python2 adding mypy error needs better error message Mar 24, 2021

frostming mentioned this issue Mar 24, 2021

Quit early when wheel fails to build #340

Merged

2 tasks

frostming closed this as completed in #340 Mar 24, 2021

This comment has been minimized.

Sign in to view

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pdm with python2 adding mypy error needs better error message #338

pdm with python2 adding mypy error needs better error message #338

linw1995 commented Mar 24, 2021

frostming commented Mar 24, 2021 •

edited

Loading

frostming commented Mar 24, 2021 •

edited

Loading

frostming commented Mar 24, 2021

linw1995 commented Mar 24, 2021

frostming commented Mar 25, 2021 •

edited

Loading

This comment has been minimized.

linw1995 commented Mar 25, 2021

frostming commented Mar 25, 2021 •

edited

Loading

linw1995 commented Mar 25, 2021

linw1995 commented Mar 25, 2021

pdm with python2 adding mypy error needs better error message #338

pdm with python2 adding mypy error needs better error message #338

Comments

linw1995 commented Mar 24, 2021

Steps to reproduce

Actual behavior

Expected behavior

Environment Information

frostming commented Mar 24, 2021 • edited Loading

frostming commented Mar 24, 2021 • edited Loading

frostming commented Mar 24, 2021

linw1995 commented Mar 24, 2021

frostming commented Mar 25, 2021 • edited Loading

This comment has been minimized.

linw1995 commented Mar 25, 2021

frostming commented Mar 25, 2021 • edited Loading

linw1995 commented Mar 25, 2021

linw1995 commented Mar 25, 2021

frostming commented Mar 24, 2021 •

edited

Loading

frostming commented Mar 24, 2021 •

edited

Loading

frostming commented Mar 25, 2021 •

edited

Loading

frostming commented Mar 25, 2021 •

edited

Loading