Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changing from allow_prereleases = true to allow_prereleases = false slows down install to a crawl #799

Closed
1 task done
weilu opened this issue Dec 15, 2021 · 2 comments · Fixed by #800
Closed
1 task done
Labels
🙋 question Further information is requested

Comments

@weilu
Copy link

weilu commented Dec 15, 2021

  • I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

Here's the pyproject.toml for replication

[project]
name = "My Project"
version = "1.0.0"
description = ""
authors = [
    {name = "Foo Bar", email = "[email protected]"},
]
dependencies = [
    "django==3.1.4",
    "django-admin-interface==0.14.1",
    "django-authtools @ git+https://github.com/luto/django-authtools.git@c5edb56",
    "pyyaml",
    "gunicorn",
    "django-heroku",
    "python-dateutil",
    "faker",
    "inflection",
    "django-storages",
    "boto3",
    "graphene-file-upload",
    "pillow",
    "django-cleanup",
    "django-graphql-jwt",
    "sentry-sdk",
    "pandas",
    "xlrd",
    "openpyxl",
    "django-auditlog @ git+https://github.com/weilu/django-auditlog.git@baa4326",
    "celery",
    "redis",
    "django-celery-beat",
    "requests",
    "posthog",
    "graphene @ git+https://github.com/weilu/graphene.git@28d3b50",
    "graphene-django @ git+https://github.com/weilu/graphene-django.git@9a1a954",
    "whoswho",
    "django-debug-toolbar",
    "django-filter"
]
requires-python = ">=3.9"
license = {text = "MIT"}

[project.urls]
homepage = ""

[tool]
[tool.pdm]
allow_prereleases = true
[[tool.pdm.source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[tool.pdm.dev-dependencies]
dev = [
    "tqdm",
    "python-docx @ git+https://github.com/savoirfairelinux/python-docx.git@feature_include_hyperlink",
    "docx2csv",
    "pytest-django",
    "responses",
    "model-bakery",
    "freezegun",
    "pytest",
]

[build-system]
requires = ["pdm-pep517"]
build-backend = "pdm.pep517.api"

Actual behavior

With allow_prereleases = true the resolution finishes in 6 minutes on my 2018 MacBook Air.
With allow_prereleases = false the resolution went on for 80 minutes on the same computer and still going (I had to kill it). When I ran with -v (with allow_prereleases = false in settings) I noticed it was repeatedly printing this snippet:

Did not find branch or tag '28d3b50', assuming revision or ref.
Preparing isolated env for PEP 517 build...
Reusing shared build env: /var/folders/x3/g3y7g_yx0v50__84lpw0dt8w0000gn/T/pdm-build-env-dyzrgpa9-shared
running egg_info
creating graphene.egg-info
writing graphene.egg-info/PKG-INFO
writing dependency_links to graphene.egg-info/dependency_links.txt
writing requirements to graphene.egg-info/requires.txt
writing top-level names to graphene.egg-info/top_level.txt
writing manifest file 'graphene.egg-info/SOURCES.txt'
reading manifest file 'graphene.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*' found under directory 'tests'
warning: no previously-included files matching '*' found under directory 'tests_py35'
warning: no previously-included files matching '*' found under directory 'examples'
adding license file 'LICENSE'
writing manifest file 'graphene.egg-info/SOURCES.txt'
running dist_info
creating /var/folders/x3/g3y7g_yx0v50__84lpw0dt8w0000gn/T/pdm-meta-1k2udn4w/graphene.egg-info
writing /var/folders/x3/g3y7g_yx0v50__84lpw0dt8w0000gn/T/pdm-meta-1k2udn4w/graphene.egg-info/PKG-INFO
writing dependency_links to /var/folders/x3/g3y7g_yx0v50__84lpw0dt8w0000gn/T/pdm-meta-1k2udn4w/graphene.egg-info/dependency_links.txt
writing requirements to /var/folders/x3/g3y7g_yx0v50__84lpw0dt8w0000gn/T/pdm-meta-1k2udn4w/graphene.egg-info/requires.txt
writing top-level names to /var/folders/x3/g3y7g_yx0v50__84lpw0dt8w0000gn/T/pdm-meta-1k2udn4w/graphene.egg-info/top_level.txt
writing manifest file '/var/folders/x3/g3y7g_yx0v50__84lpw0dt8w0000gn/T/pdm-meta-1k2udn4w/graphene.egg-info/SOURCES.txt'
reading manifest file '/var/folders/x3/g3y7g_yx0v50__84lpw0dt8w0000gn/T/pdm-meta-1k2udn4w/graphene.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*' found under directory 'tests'
warning: no previously-included files matching '*' found under directory 'tests_py35'
warning: no previously-included files matching '*' found under directory 'examples'
adding license file 'LICENSE'
writing manifest file '/var/folders/x3/g3y7g_yx0v50__84lpw0dt8w0000gn/T/pdm-meta-1k2udn4w/graphene.egg-info/SOURCES.txt'
creating '/var/folders/x3/g3y7g_yx0v50__84lpw0dt8w0000gn/T/pdm-meta-1k2udn4w/graphene.dist-info'
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
  Adding requirement Django>=1.11(from django-graphql-jwt 0.1.10)
	Found matching candidates:
	  <Candidate Django 3.2.10 from https://pypi.org/simple/django/>
	  <Candidate Django 3.2.9 from https://pypi.org/simple/django/>
	  <Candidate Django 3.2.8 from https://pypi.org/simple/django/>
	  <Candidate Django 3.2.7 from https://pypi.org/simple/django/>
	  <Candidate Django 3.2.6 from https://pypi.org/simple/django/>
	  <Candidate Django 3.2.5 from https://pypi.org/simple/django/>
	  <Candidate Django 3.2.4 from https://pypi.org/simple/django/>
	  <Candidate Django 3.2.3 from https://pypi.org/simple/django/>
	  <Candidate Django 3.2.2 from https://pypi.org/simple/django/>
	  <Candidate Django 3.2.1 from https://pypi.org/simple/django/>
	  ... [124 more candidate(s)]
  Adding requirement django-environ>=0.4.4(from django-graphql-jwt 0.1.10)
	Found matching candidates:
	  <Candidate django-environ 0.8.1 from https://pypi.org/simple/django-environ/>
	  <Candidate django-environ 0.8.0 from https://pypi.org/simple/django-environ/>
	  <Candidate django-environ 0.7.0 from https://pypi.org/simple/django-environ/>
	  <Candidate django-environ 0.6.0 from https://pypi.org/simple/django-environ/>
	  <Candidate django-environ 0.5.0 from https://pypi.org/simple/django-environ/>
	  <Candidate django-environ 0.4.5 from https://pypi.org/simple/django-environ/>
	  <Candidate django-environ 0.4.4 from https://pypi.org/simple/django-environ/>
  Adding requirement graphene>=2.0.0(from django-graphql-jwt 0.1.10)
Did not find branch or tag '28d3b50', assuming revision or ref.
...

I wonder if it's because one of the packages used a pre-release in specifying its dependency: "graphene>=3.0.0b5,<4" source? If this is the case, I guess then the question is what exactly does allow_prereleases = false mean? Is it a) "Do not fetch/recognize any pre-release packages at all" or b) "Fetch/recognize pre-release packages but do not use any pre-release packages in the resulting lock file"?

Expected behavior

If and when there's a resolution failure, fail fast instead of hanging forever?

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
PDM version:        1.11.3
Python Interpreter: /Users/weilu/.pyenv/versions/3.9.2/bin/python (3.9)
Project Root:       /Users/weilu/workspace/foo/bar
Project Packages:   /Users/weilu/workspace/foo/bar-mediation/__pypackages__/3.9
{
  "implementation_name": "cpython",
  "implementation_version": "3.9.2",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "20.5.0",
  "platform_system": "Darwin",
  "platform_version": "Darwin Kernel Version 20.5.0: Sat May  8 05:10:33 PDT 2021; root:xnu-7195.121.3~9/RELEASE_X86_64",
  "python_full_version": "3.9.2",
  "platform_python_implementation": "CPython",
  "python_version": "3.9",
  "sys_platform": "darwin"
}
@weilu weilu added the 🐛 bug Something isn't working label Dec 15, 2021
@weilu
Copy link
Author

weilu commented Dec 15, 2021

I saw the few other issues on looping & hanging and as I'm using the latest version which should have the fix in #782

@frostming
Copy link
Collaborator

Is it a) "Do not fetch/recognize any pre-release packages at all" or b) "Fetch/recognize pre-release packages but do not use any pre-release packages in the resulting lock file"?

b) is true, it excludes all prereleases even if there are no stable releases in the version range.
Just delete the allow_prereleases setting to let PDM decide automatically.

@frostming frostming added 🙋 question Further information is requested and removed 🐛 bug Something isn't working labels Dec 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🙋 question Further information is requested
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants