intermittent pip HashMismatch due to missing wheel entry in pdm.lock #596
Labels
🐛 bug
Something isn't working
Comments
|
Another thing I noticed after typing out the above is that
|
|
Hmm, the wheel for I wonder if there are multiple servers behind |
|
Yep, now the the wheel for ❯ pdm install -v
Synchronizing working set with lock file: 1 to add, 0 to update, 0 to remove
Install botocore 1.21.18 failed
Retry failed jobs
Install botocore 1.21.18 failed
ERRORS:
add botocore failed:
Traceback (most recent call last):
File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/installers/synchronizers.py", line 157, in install_candidate
installer.install(can)
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/installers/installers.py", line 43, in install
candidate.get_metadata(allow_all_wheels=False, raising=True)
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/candidates.py", line 188, in get_metadata
built = self.environment.build(ireq, self.hashes, allow_all_wheels)
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/models/environment.py", line 267, in build
downloaded = pip_shims.unpack_url(
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 239, in unpack_url
file = get_http_url(
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 104, in get_http_url
hashes.check_against_path(from_path)
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pip/_internal/utils/hashes.py", line 118, in check_against_path
return self.check_against_file(file)
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pip/_internal/utils/hashes.py", line 113, in check_against_file
return self.check_against_chunks(read_chunks(file))
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pip/_internal/utils/hashes.py", line 100, in check_against_chunks
self._raise(gots)
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pip/_internal/utils/hashes.py", line 104, in _raise
raise HashMismatch(self._allowed, gots)
pip._internal.exceptions.HashMismatch: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
unknown package:
Expected sha256 a0d64369857d86b3a6d01b0c5933671c2394584311ce3af702271ba221b09afa
Got a916f4e9ad1166cd58ee8b5373885ceceeb6cef89a03f52ec71e0008303b8db2
Traceback (most recent call last):
File "/home/anthony/.local/bin/pdm", line 8, in <module>
sys.exit(main())
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/core.py", line 185, in main
return Core().main(args)
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/core.py", line 146, in main
raise cast(Exception, err).with_traceback(traceback)
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/core.py", line 141, in main
f(options.project, options)
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/cli/commands/install.py", line 44, in handle
actions.do_sync(
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/cli/actions.py", line 166, in do_sync
handler.synchronize()
File "/home/anthony/.local/share/pdm/venv/lib/python3.8/site-packages/pdm/installers/synchronizers.py", line 324, in synchronize
raise InstallationError("Some package operations are not complete yet")
pdm.exceptions.InstallationError: Some package operations are not complete yet
[[package]]
name = "boto3"
version = "1.18.18"
requires_python = ">=3.6"
summary = "The AWS SDK for Python"
dependencies = [
"botocore<1.22.0,>=1.21.18",
"jmespath<1.0.0,>=0.7.1",
"s3transfer<0.6.0,>=0.5.0",
]
[[package]]
name = "botocore"
version = "1.21.18"
requires_python = ">=3.6"
summary = "Low-level, data-driven core of boto 3."
dependencies = [
"jmespath<1.0.0,>=0.7.1",
"python-dateutil<3.0.0,>=2.1",
"urllib3<1.27,>=1.25.4",
]
[[package]]
name = "jmespath"
version = "0.10.0"
requires_python = ">=2.6,!=3.0.*,!=3.1.*,!=3.2.*"
summary = "JSON Matching Expressions"
[[package]]
name = "python-dateutil"
version = "2.8.2"
requires_python = ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*"
summary = "Extensions to the standard Python datetime module"
dependencies = [
"six>=1.5",
]
[[package]]
name = "s3transfer"
version = "0.5.0"
requires_python = ">=3.6"
summary = "An Amazon S3 Transfer Manager"
dependencies = [
"botocore<2.0a.0,>=1.12.36",
]
[[package]]
name = "six"
version = "1.16.0"
requires_python = ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*"
summary = "Python 2 and 3 compatibility utilities"
[[package]]
name = "urllib3"
version = "1.26.6"
requires_python = ">=2.7,<4.0,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*"
summary = "HTTP library with thread-safe connection pooling, file post, and more."
[metadata]
lock_version = "3"
content_hash = "sha256:45ec19f040ec353b0bfbcaf4e6db3361d6d4913403d96b3a9986a2abb66d3b1a"
[metadata.files]
"boto3 1.18.18" = [
{file = "boto3-1.18.18-py3-none-any.whl", hash = "sha256:3faa0e4b5b6ae5bee43103477c6e141fbed61c84570cc787f2b563cfb3792342"},
{file = "boto3-1.18.18.tar.gz", hash = "sha256:325456ceae7144693bf86786b4d9c6202f46886d9d30b29780dd6a5b377f5a24"},
]
"botocore 1.21.18" = [
{file = "botocore-1.21.17.tar.gz", hash = "sha256:a0d64369857d86b3a6d01b0c5933671c2394584311ce3af702271ba221b09afa"},
]
"jmespath 0.10.0" = [
{file = "jmespath-0.10.0-py2.py3-none-any.whl", hash = "sha256:cdf6525904cc597730141d61b36f2e4b8ecc257c420fa2f4549bac2c2d0cb72f"},
{file = "jmespath-0.10.0.tar.gz", hash = "sha256:b85d0567b8666149a93172712e68920734333c0ce7e89b78b3e987f71e5ed4f9"},
]
"python-dateutil 2.8.2" = [
{file = "python_dateutil-2.8.2-py2.py3-none-any.whl", hash = "sha256:961d03dc3453ebbc59dbdea9e4e11c5651520a876d0f4db161e8674aae935da9"},
{file = "python-dateutil-2.8.2.tar.gz", hash = "sha256:0123cacc1627ae19ddf3c27a5de5bd67ee4586fbdd6440d9748f8abb483d3e86"},
]
"s3transfer 0.5.0" = [
{file = "s3transfer-0.5.0-py3-none-any.whl", hash = "sha256:9c1dc369814391a6bda20ebbf4b70a0f34630592c9aa520856bf384916af2803"},
{file = "s3transfer-0.5.0.tar.gz", hash = "sha256:50ed823e1dc5868ad40c8dc92072f757aa0e653a192845c94a3b676f4a62da4c"},
]
"six 1.16.0" = [
{file = "six-1.16.0-py2.py3-none-any.whl", hash = "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"},
{file = "six-1.16.0.tar.gz", hash = "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926"},
]
"urllib3 1.26.6" = [
{file = "urllib3-1.26.6-py2.py3-none-any.whl", hash = "sha256:39fb8672126159acb139a7718dd10806104dec1e2f0f6c88aab05d17df10c8d4"},
{file = "urllib3-1.26.6.tar.gz", hash = "sha256:f57b4c16c62fa2760b7e3d97c35b255512fb6b59a259730f36ba32ce9f8e342f"},
]
[project]
name = ""
version = ""
description = ""
authors = [
{name = "Anthony Roach", email = "[email protected]"},
]
dependencies = [
"boto3~=1.18",
]
requires-python = ">=3.8"
dynamic = ["classifiers"]
license = {text = "MIT"}
[project.urls]
homepage = ""
[build-system]
requires = ["pdm-pep517"]
build-backend = "pdm.pep517.api"
[tool]
[tool.pdm] |
|
The root cause seems to be |
|
Fixed, now the wheel won't be picked even it exists |
|
I upgraded to the latest release (v.1.8.0) which has the fix, but I'm still getting a hash mismatch: The |
|
The sha256 hash is incorrect in your lock file, it is expected to fail. This is what I got from PyPI, for the
|
|
You are right. After correcting the hash in Very strange that the hash changed, but |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Steps to reproduce
pdm initin fresh directorypdm add boto3pdm install.Actual behavior
Sometimes
pdm installwill fail with the following error:I've gotten it to happen a few times on my workstation, and it almost always fails on our Jenkins build server. It seems to be random whether
pdm installworks or not.I noticed that in
pdm.lockunder[metadata.files]botocore has only an sdist entry and no wheel entry:If I manually change this and add the wheel (file name and hash from pypi.org) then
pdm installseems to work 100% of the time. Example:Also note that in the error message the expected hash is the sdist one, but the actual hash is the wheel one. It seems like pdm is telling pip only about the sdist hash, and then pip is downloading the wheel instead of the sdist and so the hash doesn't match.
Expected behavior
pdm installshould work 100% of the time.Environment Information
On my workstation:
In Jenkins:
Note that I'm running pdm via docker in the above.
The text was updated successfully, but these errors were encountered: