-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pipenv looking in wrong index for subdependencies #5543
Comments
@CNDW Can you try add install_search_all_sources = true option to your Pipfile in the pipenv section? I am not sure that this would have worked for you in |
ah yea, my mistake. I dug through our CI history and it looks like it was The install does work when I add |
Well |
As a workaround this is 👍 It does feel like there is some inconsistency around how dependency indexes are handled. Could you shed some light on how pipenv decides what index to look in when The space has been pretty inconsistent in recent years (understandably so given how vague requirements.txt is about extra indexes) and every time something changes it's a huge timesink for myself and others to try to grok why CI mysteriously stopped working or why pipenv won't install on someone's machine while it installs fine on mine. Pipenv has been improving a lot in recent years and I'm hoping the tool can continue to bring some clarity to this space, is this something that is on the team's radar? |
When
I can appreciate this frustration--I have been working on pipenv for a year now and I can say there are so many use cases that are unaccounted for by tests and adding tests is non-trivial because the test runner already takes so long to run -- for example I have some good multi-index tests involving pytorch but that package is multiple GB of data and we cannot integrate it into the test runner. I had done work back in March /April time frame on the index restricted packages for pipenv, which is a security feature to say when we know a package is to come from a certain index we apply restrictions in the resolver to not allow to be resolved from a different source, to prevent package confusion attacks. When pip changed how the indexes were handled in the install requirements.txt files in 22.3 it seems to have caused a few issue reports like this that have all been resolved with |
@CNDW I think this issue can be closed at this point, there is ongoing work to start making the docs more granular and improving their content, it will take some time to get to where we want to be there but progress is happening. |
Issue description
When installing a package from a private registry, sub-dependencies coming from pypi are being looked for in the private package registry and as a result fail to install. This issue was not present in v2022.11.30 and was introduced with v2022.12.17
Expected result
Dependencies to be resolved correctly
Actual result
pipfile lock
completes without issue, although it does not apply aindex
to the dependencies coming frompypi
. As an aside, in the past we have had to define private subdependencies with the index, pipenv has been having trouble identifying the expected index ever since pip made the change to expect explicit indexes.The minimal example I produced has a pipfile that looks like
When I run
pipenv sync -vv
the output provides an error that looks like thispipenv is looking for
setuptools
in ourfury
indexeven though there is no index defined in the
Pipfile.lock
Steps to replicate
Create a pipfile that contains 2 registries, pypi and a 2nd with a package coming from the 2nd registry and itself having a mix of dependencies coming from pypi and the 2nd registry.
For obvious reasons, I can't share my isolated example here, but if you want my help trying to set up an example or working on this issue I am willing to spend time on it. Dependency management with out private registry has been the source of a lot of dev pain over the years and I am very motivated to help make it more consistent for all involved.
$ pipenv --support
Pipenv version:
'2022.11.30'
Pipenv location:
'/Users/travis/.local/pipx/venvs/pipenv/lib/python3.7/site-packages/pipenv'
Python location:
'/Users/travis/.local/pipx/venvs/pipenv/bin/python'
OS Name:
'posix'
User pip version:
'22.3'
user Python installations found:
3.9.6
:/usr/bin/python3
3.9.0
:/Users/travis/.pyenv/versions/3.9.0/bin/python3
3.7.4
:/Users/travis/.pyenv/versions/3.7.4/bin/python3
2.7.16
:/Users/travis/.pyenv/versions/2.7.16/bin/python
PEP 508 Information:
System environment variables:
TERM_PROGRAM
SHELL
TERM
TMPDIR
TERM_PROGRAM_VERSION
TERM_SESSION_ID
USER
SSH_AUTH_SOCK
PATH
LaunchInstanceID
__CFBundleIdentifier
PWD
XPC_FLAGS
XPC_SERVICE_NAME
SHLVL
HOME
LOGNAME
SECURITYSESSIONID
OLDPWD
GCLOUD_PROJECT
GAIN_ENV
GEMFURY_TOKEN
NPM_TOKEN
GH_TOKEN
CR_PAT
SLACK_API_TOKEN
GOOGLE_APPLICATION_CREDENTIALS
NVM_DIR
NVM_CD_FLAGS
NVM_BIN
NVM_INC
PYENV_SHELL
GAE_LIB_ROOT
GAE_PYTHONPATH
BUN_INSTALL
LANG
_
__CF_USER_TEXT_ENCODING
PIP_DISABLE_PIP_VERSION_CHECK
PIP_PYTHON_PATH
PYTHONDONTWRITEBYTECODE
PYTHONFINDER_IGNORE_UNSUPPORTED
Pipenv–specific environment variables:
Debug–specific environment variables:
PATH
:/Users/travis/.bun/bin:/Users/travis/.poetry/bin:/usr/local/opt/[email protected]/bin:/Users/travis/.pyenv/shims:/Users/travis/google-cloud-sdk/bin:/Users/travis/.nvm/versions/node/v16.18.1/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Users/travis/.pyenv/bin:/usr/local/opt/openssl/bin:/usr/local/opt/sqlite/bin:/Users/travis/.local/bin:/Users/travis/Projects/GainCompliance/pygain-cli/bin:/Users/travis/Projects/pygain-lib/bin
SHELL
:/bin/zsh
LANG
:en_US.UTF-8
PWD
:/Users/travis/Projects/tmp/pipenv-test
Contents of
Pipfile
('/Users/travis/Projects/tmp/pipenv-test/Pipfile'):Contents of
Pipfile.lock
('/Users/travis/Projects/tmp/pipenv-test/Pipfile.lock'):The text was updated successfully, but these errors were encountered: