Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal library dosen't get version from git repo #5617

Open
3 tasks done
ericman93 opened this issue May 16, 2022 · 17 comments
Open
3 tasks done

internal library dosen't get version from git repo #5617

ericman93 opened this issue May 16, 2022 · 17 comments
Labels
area/solver Related to the dependency resolver kind/bug Something isn't working as expected status/triage This issue needs to be triaged

Comments

@ericman93
Copy link

ericman93 commented May 16, 2022

  • I am on the latest Poetry version.

  • I have searched the issues of this repo and believe that this is not a duplicate.

  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

  • OS version and name: MacOS 11.1 (20C69)

  • Poetry version: 1.1.13 (tried also on 1.2.0b1)

    • Link of a Gist with the contents of your pyproject.toml file: in the issues

Issue

I have a repo A that is using 2 other repos B and C
C is also using B

A's pyproject.toml

B = {git = "<B's github repo>", branch = "my-branch"}
C = {version="^0.5.8"}

B's pyproject.toml (in my-branch branch)

[tool.poetry]
name = "B"
version = "0.31.0"

C's pyproject.toml

B = {version=">=0.30"}

The current versions of B in my PyPI server are (0.30.0 || 0.30.1 || 0.30.2 || 0.30.3 || 0.30.4)

It seems that when locking C inside A, C's not aware B's version is acceptable

  Because no versions of B match >0.30,<0.30.1 || >0.30.1,<0.30.2 || >0.30.2,<0.30.3 || >0.30.3,<0.30.4 || >0.30.4
   and B (0.30.0) depends on B (0.30.0), B (>=0.30,<0.30.1 || >0.30.1,<0.30.2 || >0.30.2,<0.30.3 || >0.30.3,<0.30.4 || >0.30.4) requires B (0.30.0).
  And because B (0.30.1) depends on B (0.30.1)
   and B (0.30.2) depends on B (0.30.2), B (>=0.30,<0.30.3 || >0.30.3,<0.30.4 || >0.30.4) requires B (0.30.0 || 0.30.1 || 0.30.2).
  And because B (0.30.3) depends on B (0.30.3)
   and B (0.30.4) depends on B (0.30.4), B (>=0.30) requires B (0.30.0 || 0.30.1 || 0.30.2 || 0.30.3 || 0.30.4).
  Because no versions of C match >0.5.8
   and C (0.5.8) depends on B (>=0.30), C (>=0.5.8) requires B (>=0.30).
  Thus, C (>=0.5.8) requires B (0.30.0 || 0.30.1 || 0.30.2 || 0.30.3 || 0.30.4).
  And because B (rev my-branch) depends on B (rev my-branch), C (>=0.5.8) is incompatible with B (rev my-branch).
  So, because A depends on both B (branch my-branch) and C (>=0.5.8), version solving failed.

I tried to publish my-branch as a beta branch, changed A's to use this version, and still had the same issue

@ericman93 ericman93 added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels May 16, 2022
@ericman93 ericman93 changed the title internal library dosen't get github version internal library dosen't get version from git repo May 16, 2022
@radoering
Copy link
Member

May be related to python-poetry/poetry-core#325 and an incorrect call of is_same_package_as() in a place where only the names of the packages should be compared.

@abn
Copy link
Member

abn commented May 18, 2022

@ericman93 Can you please try the version on master?

Using pipx

pipx install --suffix=@git --force 'poetry @ git+https://github.com/python-poetry/poetry.git'
poetry@git lock
poetry@git install

Using a container (podman | docker)

podman run --rm -i --entrypoint bash docker.io/python:3.10 <<EOF
set -xe
python -m pip install --disable-pip-version-check -q git+https://github.com/python-poetry/poetry.git
# try your poetry commands here
EOF

@abn abn added the status/waiting-on-response Waiting on response from author label May 19, 2022
@ericman93
Copy link
Author

ericman93 commented May 20, 2022

I got this error

Updating dependencies
Resolving dependencies... (36.8s)

  AssertionError

  at ~/Library/Caches/pypoetry/virtualenvs/A-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:151 in _register
      147│         name = assignment.dependency.complete_name
      148│         old_positive = self._positive.get(name)
      149│         if old_positive is not None:
      150│             value = old_positive.intersect(assignment)
    → 151│             assert value is not None
      152│             self._positive[name] = value
      153│
      154│             return
      155│

By the way, before having the final error I had got

  404 Client Error: Not Found for url: https://pypi.org/pypi/D/json/

D is my lib that exists in another poetry source
The source of D wasn't mentioned in pyproject.toml
After adding source key it worked (but failed on the above)
It used to work before without adding the source, is that a new feature? to look only in the specified source?

@abn
Copy link
Member

abn commented May 26, 2022

@ericman93 I think that is #5682

@ericman93
Copy link
Author

#5782 relates to the 404 (not an issue, just asked if this is a new feature) or to the AssertionError?

@abn
Copy link
Member

abn commented May 27, 2022

Ah I misunderstood your comment.

The assertion error looks like it's the solver, that scenario should work but will have to look more closely. Do you have the -vvv logs with the latest master?

@ericman93
Copy link
Author

  Stack trace:

  17  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/cleo/application.py:330 in run
       328│
       329│             try:
     → 330│                 exit_code = self._run(io)
       331│             except Exception as e:
       332│                 if not self._catch_exceptions:

  16  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/console/application.py:182 in _run
       180│         self._load_plugins(io)
       181│
     → 182│         exit_code: int = super()._run(io)
       183│         return exit_code
       184│

  15  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/cleo/application.py:425 in _run
       423│                 io.set_input(ArgvInput(argv))
       424│
     → 425│         exit_code = self._run_command(command, io)
       426│         self._running_command = None
       427│

  14  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/cleo/application.py:467 in _run_command
       465│
       466│         if error is not None:
     → 467│             raise error
       468│
       469│         return event.exit_code

  13  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/cleo/application.py:451 in _run_command
       449│
       450│             if event.command_should_run():
     → 451│                 exit_code = command.run(io)
       452│             else:
       453│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  12  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/cleo/commands/base_command.py:118 in run
       116│         io.input.validate()
       117│
     → 118│         status_code = self.execute(io)
       119│
       120│         if status_code is None:

  11  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/cleo/commands/command.py:85 in execute
        83│
        84│         try:
     →  85│             return self.handle()
        86│         except KeyboardInterrupt:
        87│             return 1

  10  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/console/commands/update.py:55 in handle
        53│         self._installer.update(True)
        54│
     →  55│         return self._installer.run()
        56│

   9  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/installation/installer.py:112 in run
       110│         local_repo = Repository()
       111│
     → 112│         return self._do_install(local_repo)
       113│
       114│     def dry_run(self, dry_run: bool = True) -> Installer:

   8  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/installation/installer.py:245 in _do_install
       243│                 source_root=self._env.path.joinpath("src")
       244│             ):
     → 245│                 ops = solver.solve(use_latest=self._whitelist).calculate_operations()
       246│         else:
       247│             self._io.write_line("Installing dependencies from lock file")

   7  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/puzzle/solver.py:80 in solve
        78│         with self._provider.progress():
        79│             start = time.time()
     →  80│             packages, depths = self._solve(use_latest=use_latest)
        81│             end = time.time()
        82│

   6  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/puzzle/solver.py:149 in _solve
       147│
       148│         try:
     → 149│             result = resolve_version(
       150│                 self._package, self._provider, locked=locked, use_latest=use_latest
       151│             )

   5  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/mixology/__init__.py:24 in resolve_version
        22│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
        23│
     →  24│     return solver.solve()
        25│

   4  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/mixology/version_solver.py:122 in solve
       120│             while next is not None:
       121│                 self._propagate(next)
     → 122│                 next = self._choose_package_version()
       123│
       124│             return self._result()

   3  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/mixology/version_solver.py:463 in _choose_package_version
       461│
       462│         if not conflict:
     → 463│             self._solution.decide(package.package)
       464│             self._log(
       465│                 f"selecting {package.complete_name} ({package.full_pretty_version})"

   2  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:90 in decide
        88│         self._decisions[package.complete_name] = package
        89│
     →  90│         self._assign(
        91│             Assignment.decision(package, self.decision_level, len(self._assignments))
        92│         )

   1  ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:115 in _assign
       113│         """
       114│         self._assignments.append(assignment)
     → 115│         self._register(assignment)
       116│
       117│     def backtrack(self, decision_level: int) -> None:

  AssertionError



  at ~/Library/Caches/pypoetry/virtualenvs/cookbook-sCL0SNDv-py3.8/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:151 in _register
      147│         name = assignment.dependency.complete_name
      148│         old_positive = self._positive.get(name)
      149│         if old_positive is not None:
      150│             value = old_positive.intersect(assignment)
    → 151│             assert value is not None
      152│             self._positive[name] = value
      153│
      154│             return
      155│

@abn abn added area/solver Related to the dependency resolver and removed status/waiting-on-response Waiting on response from author labels May 27, 2022
@abn
Copy link
Member

abn commented May 27, 2022

Maybe also the bits above the stack trace as well.

@ericman93
Copy link
Author

Source (PyPI): 17 packages found for coverage >=5.2.1
[urllib3.connectionpool] https://pypi.fury.io:443 "GET /explorium/coverage/ HTTP/1.1" 404 10
   1: selecting coverage (6.4)
Source (PyPI): No release information found for py-0.8.0-alpha2, skipping
Source (PyPI): No release information found for py-0.9.0, skipping
Source (PyPI): No release information found for py-1.4.32.dev1, skipping
Source (PyPI): 4 packages found for py >=1.8.2
[urllib3.connectionpool] https://pypi.fury.io:443 "GET /explorium/py/ HTTP/1.1" 404 10
   1: selecting py (1.11.0)
Source (PyPI): 4 packages found for pluggy >=0.12,<2.0
[urllib3.connectionpool] https://pypi.fury.io:443 "GET /explorium/pluggy/ HTTP/1.1" 404 10
   1: selecting pluggy (1.0.0)
Source (PyPI): No release information found for iniconfig-0.0, skipping
Source (PyPI): 5 packages found for iniconfig *
[urllib3.connectionpool] https://pypi.fury.io:443 "GET /explorium/iniconfig/ HTTP/1.1" 404 10
   1: selecting iniconfig (1.1.1)
Source (PyPI): No release information found for numpy-0.9.6, skipping
Source (PyPI): No release information found for numpy-0.9.8, skipping
Source (PyPI): No release information found for numpy-1.0.3, skipping
Source (PyPI): No release information found for numpy-1.0.4, skipping
Source (PyPI): No release information found for numpy-1.0b1, skipping
Source (PyPI): No release information found for numpy-1.0b4, skipping
Source (PyPI): No release information found for numpy-1.0b5, skipping
Source (PyPI): No release information found for numpy-1.0rc1, skipping
Source (PyPI): No release information found for numpy-1.0rc2, skipping
Source (PyPI): No release information found for numpy-1.0rc3, skipping
Source (PyPI): No release information found for numpy-1.1.1, skipping
Source (PyPI): No release information found for numpy-1.2.0, skipping
Source (PyPI): No release information found for numpy-1.2.1, skipping
Source (PyPI): No release information found for numpy-1.4.0, skipping
Source (PyPI): 53 packages found for numpy >=1.14
[urllib3.connectionpool] https://pypi.fury.io:443 "GET /explorium/numpy/ HTTP/1.1" 404 10
   1: selecting numpy (1.22.4)
Source (PyPI): 1 packages found for sklearn 0.0
[urllib3.connectionpool] https://pypi.fury.io:443 "GET /explorium/sklearn/ HTTP/1.1" 404 10
   1: fact: sklearn (0.0) depends on scikit-learn (*)
   1: selecting sklearn (0.0)
Source (PyPI): No release information found for scipy-0.4.4, skipping
Source (PyPI): No release information found for scipy-0.5.2, skipping
Source (PyPI): No release information found for scipy-0.6.0, skipping
Source (PyPI): No release information found for scipy-0.7.0, skipping
Source (PyPI): No release information found for scipy-0.7.2, skipping
Source (PyPI): 1 packages found for scipy 1.7.0
[urllib3.connectionpool] https://pypi.fury.io:443 "GET /explorium/scipy/ HTTP/1.1" 404 10
   1: fact: scipy (1.7.0) depends on numpy (>=1.16.5,<1.23.0)
   1: selecting scipy (1.7.0)
Source (PyPI): No release information found for scikit-learn-0.14a1, skipping
Source (PyPI): 2 packages found for scikit-learn 0.22.2
[urllib3.connectionpool] https://pypi.fury.io:443 "GET /explorium/scikit-learn/ HTTP/1.1" 404 10
   1: fact: scikit-learn (0.22.2.post1) depends on numpy (>=1.11.0)
   1: fact: scikit-learn (0.22.2.post1) depends on scipy (>=0.17.0)
   1: fact: scikit-learn (0.22.2.post1) depends on joblib (>=0.11)
   1: Version solving took 22.211 seconds.
   1: Tried 1 solutions.

  Stack trace:

@ronlut
Copy link

ronlut commented May 29, 2022

@abn Happens for me without any internal library

  Stack trace:

  4  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/puzzle/solver.py:151 in _solve
      149│ 
      150│         try:
    → 151│             result = resolve_version(
      152│                 self._package, self._provider, locked=locked, use_latest=use_latest
      153│             )

  3  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/mixology/__init__.py:24 in resolve_version
       22│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
       23│ 
    →  24│     return solver.solve()
       25│ 

  2  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/mixology/version_solver.py:125 in solve
      123│             while next is not None:
      124│                 self._propagate(next)
    → 125│                 next = self._choose_package_version()
      126│ 
      127│             return self._result()

  1  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/mixology/version_solver.py:448 in _choose_package_version
      446│             package = locked
      447│ 
    → 448│         package = self._provider.complete_package(package)
      449│ 
      450│         conflict = False

  OverrideNeeded

  ({Package('pandas', '1.4.2'): {'numpy': <Dependency numpy (>=1.18.5)>}}, {Package('pandas', '1.4.2'): {'numpy': <Dependency numpy (>=1.19.2)>}}, {Package('pandas', '1.4.2'): {'numpy': <Dependency numpy (>=1.20.0)>}})

  at ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/puzzle/provider.py:765 in complete_package
      761│                     current_overrides.update({package: package_overrides})
      762│                     overrides.append(current_overrides)
      763│ 
      764│             if overrides:
    → 765│                 raise OverrideNeeded(*overrides)
      766│ 
      767│         # Modifying dependencies as needed
      768│         clean_dependencies = []
      769│         for dep in dependencies:

The following error occurred when trying to handle this error:


  Stack trace:

  19  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:330 in run
       328│ 
       329│             try:
     → 330│                 exit_code = self._run(io)
       331│             except Exception as e:
       332│                 if not self._catch_exceptions:

  18  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/console/application.py:182 in _run
       180│         self._load_plugins(io)
       181│ 
     → 182│         exit_code: int = super()._run(io)
       183│         return exit_code
       184│ 

  17  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:425 in _run
       423│                 io.set_input(ArgvInput(argv))
       424│ 
     → 425│         exit_code = self._run_command(command, io)
       426│         self._running_command = None
       427│ 

  16  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:467 in _run_command
       465│ 
       466│         if error is not None:
     → 467│             raise error
       468│ 
       469│         return event.exit_code

  15  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/application.py:451 in _run_command
       449│ 
       450│             if event.command_should_run():
     → 451│                 exit_code = command.run(io)
       452│             else:
       453│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  14  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/commands/base_command.py:118 in run
       116│         io.input.validate()
       117│ 
     → 118│         status_code = self.execute(io)
       119│ 
       120│         if status_code is None:

  13  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/cleo/commands/command.py:85 in execute
        83│ 
        84│         try:
     →  85│             return self.handle()
        86│         except KeyboardInterrupt:
        87│             return 1

  12  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/console/commands/update.py:55 in handle
        53│         self._installer.update(True)
        54│ 
     →  55│         return self._installer.run()
        56│ 

  11  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/installer.py:112 in run
       110│         local_repo = Repository()
       111│ 
     → 112│         return self._do_install(local_repo)
       113│ 
       114│     def dry_run(self, dry_run: bool = True) -> Installer:

  10  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/installation/installer.py:245 in _do_install
       243│                 source_root=self._env.path.joinpath("src")
       244│             ):
     → 245│                 ops = solver.solve(use_latest=self._whitelist).calculate_operations()
       246│         else:
       247│             self._io.write_line("Installing dependencies from lock file")

   9  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/puzzle/solver.py:82 in solve
        80│         with self._provider.progress():
        81│             start = time.time()
     →  82│             packages, depths = self._solve(use_latest=use_latest)
        83│             end = time.time()
        84│ 

   8  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/puzzle/solver.py:157 in _solve
       155│             packages = result.packages
       156│         except OverrideNeeded as e:
     → 157│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
       158│         except SolveFailure as e:
       159│             raise SolverProblemError(e)

   7  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/puzzle/solver.py:116 in solve_in_compatibility_mode
       114│             )
       115│             self._provider.set_overrides(override)
     → 116│             _packages, _depths = self._solve(use_latest=use_latest)
       117│             for index, package in enumerate(_packages):
       118│                 if package not in packages:

   6  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/puzzle/solver.py:151 in _solve
       149│ 
       150│         try:
     → 151│             result = resolve_version(
       152│                 self._package, self._provider, locked=locked, use_latest=use_latest
       153│             )

   5  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/mixology/__init__.py:24 in resolve_version
        22│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
        23│ 
     →  24│     return solver.solve()
        25│ 

   4  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/mixology/version_solver.py:125 in solve
       123│             while next is not None:
       124│                 self._propagate(next)
     → 125│                 next = self._choose_package_version()
       126│ 
       127│             return self._result()

   3  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/mixology/version_solver.py:466 in _choose_package_version
       464│ 
       465│         if not conflict:
     → 466│             self._solution.decide(package.package)
       467│             self._log(
       468│                 f"selecting {package.complete_name} ({package.full_pretty_version})"

   2  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:90 in decide
        88│         self._decisions[package.complete_name] = package
        89│ 
     →  90│         self._assign(
        91│             Assignment.decision(package, self.decision_level, len(self._assignments))
        92│         )

   1  ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:115 in _assign
       113│         """
       114│         self._assignments.append(assignment)
     → 115│         self._register(assignment)
       116│ 
       117│     def backtrack(self, decision_level: int) -> None:

  AssertionError

  

  at ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:151 in _register
      147│         name = assignment.dependency.complete_name
      148│         old_positive = self._positive.get(name)
      149│         if old_positive is not None:
      150│             value = old_positive.intersect(assignment)
    → 151│             assert value is not None
      152│             self._positive[name] = value
      153│ 
      154│             return
      155│ 

@dimbleby
Copy link
Contributor

@ronlut if you have a repro that does not require private libraries and repositories please share your pyproject.toml and poetry.lock so that maintainers have a shot at debugging.

@ronlut
Copy link

ronlut commented May 29, 2022

@dimbleby Thanks for the comment, actually now I tried pinning down the problem and it stops working as soon as I even add a private registry (source), without any library from the private registry.

What interesting though, is if I remove the secondary=true line, delete the poetry.lock and rerun poetry update everything works.

So I think the problem is a declaration of a private registry with secondary=true.

EDIT: actually, I think it is even not consistent. I ran poetry update 3 times with private source and secondary=true (without any libraries) and it failed 2 times out of 3.
With secondary=false, sometimes it works sometimes it doesn't.

@dimbleby
Copy link
Contributor

dimbleby commented Jun 7, 2022

absent a way of reproducing this, we can guess that it's the same as #5791 (which did give a repro)

@ericman93
Copy link
Author

tried with main version
changed albus to GitHub version that its version is 0.37.1

    Because no versions of enrichment-catalog match >0.6,<0.6.1 || >0.6.1,<0.6.2 || >0.6.2,<0.6.3 || >0.6.3,<0.6.4 || >0.6.4,<0.6.5 || >0.6.5,<0.6.6 || >0.6.6,<0.7
 and enrichment-catalog (0.6.0) depends on albus (>=0.30.0), enrichment-catalog (>=0.6,<0.6.1 || >0.6.1,<0.6.2 || >0.6.2,<0.6.3 || >0.6.3,<0.6.4 || >0.6.4,<0.6.5 || >0.6.5,<0.6.6 || >0.6.6,<0.7) requires albus (>=0.30.0).
    And because enrichment-catalog (0.6.1) depends on albus (>=0.30.0)
 and enrichment-catalog (0.6.2) depends on albus (>=0.30.0), enrichment-catalog (>=0.6,<0.6.3 || >0.6.3,<0.6.4 || >0.6.4,<0.6.5 || >0.6.5,<0.6.6 || >0.6.6,<0.7) requires albus (>=0.30.0).
    And because enrichment-catalog (0.6.3) depends on albus (>=0.30.0)
 and enrichment-catalog (0.6.4) depends on albus (>=0.30.0), enrichment-catalog (>=0.6,<0.6.5 || >0.6.5,<0.6.6 || >0.6.6,<0.7) requires albus (>=0.30.0).
    And because enrichment-catalog (0.6.5) depends on albus (>=0.30.0)
 and enrichment-catalog (0.6.6) depends on albus (>=0.30.0), enrichment-catalog (>=0.6,<0.7) requires albus (>=0.30.0).
    And because albus (0.37.1 git rev run-sync-using-anyio) is incompatible with albus (>=0.30.0) (1), albus (0.37.1 git rev run-sync-using-anyio) is incompatible with enrichment-catalog (>=0.6,<0.7)
    So, because chef depends on both albus (0.37.1 git branch run-sync-using-anyio) and enrichment_catalog (^0.6), version solving failed.

after deleting poetry.lock and running poetry update again it finished in 55 seconds but nothing changed
I deleted the lock again, and ran poetry update -vvv, and I got the same issue as before
full poetry lock logs are attached
poetry.log

@dimbleby
Copy link
Contributor

dimbleby commented Jun 8, 2022

You still have not provided a way to reproduce this.

I see that your log has reached scikit-learn 0.22.2.post1 when it falls over, the post-release here is consistent with this being the same thing as #5791

NB the fix was in poetry-core, not poetry, so if you are trying to test the fix it is poetry-core that you will need to update

@ericman93
Copy link
Author

it finally ran until Writing lock file part, but it crashed

Writing lock file

Finding the necessary packages for the current system

  Stack trace:

  17  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/cleo/application.py:329 in run
       327│
       328│             try:
     → 329│                 exit_code = self._run(io)
       330│             except Exception as e:
       331│                 if not self._catch_exceptions:

  16  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/console/application.py:188 in _run
       186│         self._load_plugins(io)
       187│
     → 188│         exit_code: int = super()._run(io)
       189│         return exit_code
       190│

  15  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/cleo/application.py:423 in _run
       421│             io.input.set_stream(stream)
       422│
     → 423│         exit_code = self._run_command(command, io)
       424│         self._running_command = None
       425│

  14  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/cleo/application.py:465 in _run_command
       463│
       464│         if error is not None:
     → 465│             raise error
       466│
       467│         return event.exit_code

  13  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/cleo/application.py:449 in _run_command
       447│
       448│             if event.command_should_run():
     → 449│                 exit_code = command.run(io)
       450│             else:
       451│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  12  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/cleo/commands/base_command.py:119 in run
       117│         io.input.validate()
       118│
     → 119│         status_code = self.execute(io)
       120│
       121│         if status_code is None:

  11  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/cleo/commands/command.py:83 in execute
        81│
        82│         try:
     →  83│             return self.handle()
        84│         except KeyboardInterrupt:
        85│             return 1

  10  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/console/commands/update.py:55 in handle
        53│         self._installer.update(True)
        54│
     →  55│         return self._installer.run()
        56│

   9  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/installation/installer.py:111 in run
       109│             self._execute_operations = False
       110│
     → 111│         return self._do_install()
       112│
       113│     def dry_run(self, dry_run: bool = True) -> Installer:

   8  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/installation/installer.py:309 in _do_install
       307│
       308│         with solver.use_environment(self._env):
     → 309│             ops = solver.solve(use_latest=self._whitelist).calculate_operations(
       310│                 with_uninstalls=self._requires_synchronization,
       311│                 synchronize=self._requires_synchronization,

   7  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/puzzle/solver.py:78 in solve
        76│         with self._provider.progress():
        77│             start = time.time()
     →  78│             packages, depths = self._solve(use_latest=use_latest)
        79│             end = time.time()
        80│

   6  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/puzzle/solver.py:147 in _solve
       145│
       146│         try:
     → 147│             result = resolve_version(
       148│                 self._package, self._provider, locked=locked, use_latest=use_latest
       149│             )

   5  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/mixology/__init__.py:24 in resolve_version
        22│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
        23│
     →  24│     return solver.solve()
        25│

   4  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/mixology/version_solver.py:125 in solve
       123│             while next is not None:
       124│                 self._propagate(next)
     → 125│                 next = self._choose_package_version()
       126│
       127│             return self._result()

   3  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/mixology/version_solver.py:466 in _choose_package_version
       464│
       465│         if not conflict:
     → 466│             self._solution.decide(package.package)
       467│             self._log(
       468│                 f"selecting {package.complete_name} ({package.full_pretty_version})"

   2  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:90 in decide
        88│         self._decisions[package.complete_name] = package
        89│
     →  90│         self._assign(
        91│             Assignment.decision(package, self.decision_level, len(self._assignments))
        92│         )

   1  ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:115 in _assign
       113│         """
       114│         self._assignments.append(assignment)
     → 115│         self._register(assignment)
       116│
       117│     def backtrack(self, decision_level: int) -> None:

  AssertionError



  at ~/Library/Caches/pypoetry/virtualenvs/chef-HJnWYQhJ-py3.8/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:151 in _register
      147│         name = assignment.dependency.complete_name
      148│         old_positive = self._positive.get(name)
      149│         if old_positive is not None:
      150│             value = old_positive.intersect(assignment)
    → 151│             assert value is not None
      152│             self._positive[name] = value
      153│
      154│             return
      155│

and after running it again it failed for different reason

  ({Package('pandas', '1.4.2'): {'numpy': <Dependency numpy (>=1.18.5)>}}, {Package('pandas', '1.4.2'): {'numpy': <Dependency numpy (>=1.19.2)>}}, {Package('pandas', '1.4.2'): {'numpy': <Dependency numpy (>=1.20.0)>}})

I'm trying to reproduce it with open repo but with no luck
I have multiple internal repos that are using more internal repos from my gemfury

@ericman93
Copy link
Author

ericman93 commented Jun 23, 2022

I removed the problematic sckitlearn package and now the solving is finishes successfully 100% of the time
the issue now is that it can't get a valid version of uvicorn (albus is from my private repo)

  Because no versions of albus match >=0.40,<0.40.2 || >0.40.2,<0.41
   and albus (0.40.2) depends on uvicorn (>=0.17.0,<0.18.0), albus (>=0.40,<0.41) requires uvicorn (>=0.17.0,<0.18.0).
  So, because no versions of uvicorn match >=0.17.0,<0.18.0
   and chef depends on albus (^0.40), version solving failed.

but reading the -vvv logs I do see that it found a valid version

   1: selecting uvicorn[standard] (0.17.6)

right after locking the file, I see those logs (and those are the only libs with extras)

Finding the necessary packages for the current system
Falling back to installed packages to discover metadata for uvicorn[standard]
Found 0 compatible packages for uvicorn[standard]
Falling back to installed packages to discover metadata for dask[distributed]
Found 0 compatible packages for dask[distributed]
Falling back to installed packages to discover metadata for albus[aioredis,atlas,dask,explorium,filesystem,http,interops,jupyter,message_bus,pandas,rq,s3,tracing]
Found 0 compatible packages for albus[
aioredis,atlas,dask,explorium,filesystem,http,interops,jupyter,message_bus,pandas,rq,s3,tracing]

Albus's pyproject

[tool.poetry.dependencies]
uvicorn = { version = "^0.17.0", optional = true, extras=['standard'] }
dask = { version = "==2022.5.0", extras = ["distributed"], optional = true }

[tool.poetry.extras]
http = ["gunicorn", "uvicorn", "fastapi", "python-jose", "auth0-python", "inflect", "jaraco.text", "opentelemetry-instrumentation-fastapi"]
dask = ["dask", "crontab", "bokeh"]

pyproject of the repo I'm trying to update

[tool.poetry.dependencies]
albus = { version = "^0.40", source="explorium", extras = ["http", "pandas", "filesystem", "s3", "message_bus", "explorium", "rq", "aioredis", "atlas", "interops", "dask", "jupyter", "tracing"] }

[tool.poetry.dev-dependencies]
albus = { version = "^0.40", source="explorium", extras = ["testing"] }

full logs are attached
poetry.log

@dimbleby still no repo but a major update

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/solver Related to the dependency resolver kind/bug Something isn't working as expected status/triage This issue needs to be triaged
Projects
None yet
Development

No branches or pull requests

5 participants