Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uv pip install --require-hashes fails when (transitive) dependencies aren't pinned to latest version #10425

Closed
cpswan opened this issue Jan 9, 2025 · 5 comments · Fixed by #10441 or #10833
Assignees
Labels
bug Something isn't working

Comments

@cpswan
Copy link

cpswan commented Jan 9, 2025

uv version 0.5.16

Description

When I uv pip install --require-hashes -r requirements.txt I get:

⠙ urllib3==2.3.0
error: In `--require-hashes` mode, all requirements must be pinned upfront with `==`, but found: `urllib3`

But my requirements.txt does specify a version and hashes for urllib3:

urllib3==2.2.3 ; python_version >= "3.8" \
    --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \
    --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9

This doesn't happen if I have a requirements.txt that pins to urllib3==2.3.0, which is the latest version.

Hypothesis

The dependency solver is trying to pull in the latest version of urllib3 rather than the version pinned in requirements.txt, and (obviously) not finding pinning (or hashes) for that version.

How I got here (reproduction)

I start out with a pyproject.toml:

[project]
name = "test-require-hashes"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
    "requests>=2.32.3",
]

Then generate a poetry.lock file with poetry lock. Inside that I can see:

[[package]]
name = "requests"
version = "2.32.3"
description = "Python HTTP for Humans."
optional = false
python-versions = ">=3.8"
groups = ["main"]
files = [
    {file = "requests-2.32.3-py3-none-any.whl", hash = "sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6"},
    {file = "requests-2.32.3.tar.gz", hash = "sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760"},
]

NB I'm using poetry 2.0.0 here as it respects PEP 621. I've also recreated this with poetry 1.8.2, but the pyproject.toml needs to use the old style [tool.poetry] rather than [project].

Then generate a requirements.txt from the poetry.lock with poetry export --output requirements.txt, which contains:

urllib3==2.2.3 ; python_version >= "3.8" \
    --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \
    --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9

Finally ask uv to install from that requirements.txt with --require-hashes uv pip install --require-hashes -r requirements.txt

Python 3.8 is deprecated

Yes, I know, if I specify requires-python = ">=3.9" then I get a poetry.lock and requirements.txt that use urllib3==2.3.0

Whilst using 3.8 seems to be what surfaced this issue for me I don't think it's the fundamental cause.

@cpswan
Copy link
Author

cpswan commented Jan 9, 2025

I meant to also add the --verbose logs, which ends:

<snip lines 1-76>
DEBUG Searching for a compatible version of urllib3 (>=1.21.1, <3)
DEBUG Found installed version of urllib3==2.3.0 that satisfies >=1.21.1, <3
DEBUG Selecting: urllib3==2.3.0 [installed] (installed)
DEBUG Released lock at `/home/chris/testing/uv_require_hashes/test_require_hashes/.venv/.lock`
error: In `--require-hashes` mode, all requirements must be pinned upfront with `==`, but found: `urllib3`

Other mentions of urllib3:

<snip lines 1-5>
DEBUG At least one requirement is not satisfied: urllib3==2.2.3 ; python_full_version >= '3.8'
<snip lines 7-13>
DEBUG Adding direct dependency: urllib3{python_full_version >= '3.8'}>=2.2.3, <2.2.3+
<snip lines 15-55>
DEBUG Found fresh response for: https://pypi.org/simple/urllib3/
<snip lines 57-66>
DEBUG Adding transitive dependency for requests==2.32.3: urllib3>=1.21.1, <3
<snip lines 68-71>
DEBUG Found installed version of urllib3==2.3.0 that satisfies >=1.21.1, <3
<snip lines 73-74>
DEBUG Adding transitive dependency for requests==2.32.3: urllib3>=1.21.1, <3
<snip lines 76-81

So the puzzle here is how Adding direct dependency: urllib3{python_full_version >= '3.8'}>=2.2.3, <2.2.3+ isn't used to satisfy Adding transitive dependency for requests==2.32.3: urllib3>=1.21.1, <3 and instead it goes off on a search that ends with Found installed version of urllib3==2.3.0 that satisfies >=1.21.1, <3

@charliermarsh
Copy link
Member

Is this the entire requirements.txt file?

urllib3==2.2.3 ; python_version >= "3.8" \
    --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \
    --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9

@cpswan
Copy link
Author

cpswan commented Jan 9, 2025

Is this the entire requirements.txt file?

No, I linked to a gist with the entire requirements.txt file as pasting huge blobs of text into issues distracts from the narrative.

@charliermarsh
Copy link
Member

Thanks! Missed it.

@charliermarsh charliermarsh self-assigned this Jan 9, 2025
@charliermarsh charliermarsh added the bug Something isn't working label Jan 9, 2025
@cpswan
Copy link
Author

cpswan commented Jan 13, 2025

Thanks for the quick resolution on this @charliermarsh :)

charliermarsh added a commit that referenced this issue Jan 22, 2025
## Summary

This PR reverts #10441 and applies a
different fix for #10425.

In #10441, I changed prioritization to visit proxies eagerly. I think
this is actually wrong, since it means we prioritize proxy packages
above _everything_ else. And while a proxy only depends on itself, it
does mean we're selecting a _version_ for the proxy package earlier than
anything else. So, if you look at #10828, we end up choosing a version
for `async-timeout` before we choose a version for `langchain`, despite
the latter being a first-party dependency. (`async-timeout` has a marker
on it, so it has a proxy package, so we solve for it first.)

To fix #10425, we instead need to make sure we visit proxies in the
order we see them. I think the virtual tiebreaker for proxies is
reversed? We want to visit the package we see first, first.

So, in short: this reverts #10441, then corrects the ordering for
visiting proxies.

Closes #10828.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants