Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setuptools backend does not find PROJECT_ROOT dependencies #1927

Closed
hannahdiels-ontoforce opened this issue May 15, 2023 · 5 comments
Closed
Labels
🐛 bug Something isn't working

Comments

@hannahdiels-ontoforce
Copy link

Basically the same as #1658 but it's closed and we do need setuptools to build a C extension so we can't use the workaround.

Steps to reproduce

[project]
name = "foo"
version = ""
description = ""
authors = []
dependencies = ["bar @ file:///${PROJECT_ROOT}/../bar"]
requires-python = "==3.11.*"

[build-system]
# Not pdm because we need to build C extensions.
requires = ["setuptools>=61", "wheel"]
build-backend = "setuptools.build_meta"

pdm lock -vv

Actual behavior

$ pdm lock -vv
STATUS: Resolving dependencies
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   bar @ file:///${PROJECT_ROOT}/../bar
pdm.termui:   python>=3.11,<3.12
pdm.termui:   Adding requirement bar @ file:///${PROJECT_ROOT}/../bar
pdm.termui:   Adding requirement python>=3.11,<3.12
pdm.termui: ======== Starting round 0 ========
Traceback (most recent call last):
  File "/home/Tim.Diels/.local/bin/pdm", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/core.py", line 268, in main
    return Core().main(args)
           ^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/core.py", line 192, in main
    raise cast(Exception, err).with_traceback(traceback) from None
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/core.py", line 187, in main
    self.handle(project, options)
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/core.py", line 153, in handle
    command.handle(project, options)
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/cli/commands/lock.py", line 61, in handle
    actions.do_lock(
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/cli/actions.py", line 104, in do_lock
    mapping, dependencies = resolve(
                            ^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/resolver/core.py", line 35, in resolve
    result = resolver.resolve(requirements, max_rounds)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/resolvelib/resolvers.py", line 426, in resolve
    name = min(unsatisfied_names, key=self._get_preference)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/resolvelib/resolvers.py", line 203, in _get_preference
    return self._p.get_preference(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/resolver/providers.py", line 85, in get_preference
    is_backtrack_cause = any(dep.identify() in backtrack_identifiers for dep in self.get_dependencies(candidate))
                                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/resolver/providers.py", line 182, in get_dependencies
    deps, requires_python, _ = self.repository.get_dependencies(candidate)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/models/repositories.py", line 82, in get_dependencies
    requirements, requires_python, summary = getter(candidate)
                                             ^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/models/repositories.py", line 41, in wrapper
    result = func(self, candidate)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/models/repositories.py", line 213, in _get_dependencies_from_metadata
    deps = prepared.get_dependencies_from_metadata()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/models/candidates.py", line 510, in get_dependencies_from_metadata
    self.req.project_name, self.metadata.requires or [], extras  # type: ignore[arg-type]
                           ^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/models/candidates.py", line 496, in metadata
    result = self.prepare_metadata()
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/models/candidates.py", line 425, in prepare_metadata
    self.obtain(allow_all=True)
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/pdm/models/candidates.py", line 417, in obtain
    result = finder.download_and_unpack(self.link, build_dir, download_dir, hash_options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/unearth/finder.py", line 391, in download_and_unpack
    file = unpack_link(
           ^^^^^^^^^^^^
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/unearth/preparer.py", line 295, in unpack_link
    validator.validate_path(artifact)
  File "/home/Tim.Diels/.local/share/pdm/venv/lib/python3.11/site-packages/unearth/preparer.py", line 116, in validate_path
    with path.open("rb") as f:
         ^^^^^^^^^^^^^^^
  File "/home/Tim.Diels/.install/pyenv/versions/3.11.1/lib/python3.11/pathlib.py", line 1044, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/${PROJECT_ROOT}/../bar'

Expected behavior

It works with pdm-backend, but with the setuptools backend it apparently does not substitute ${PROJECT_ROOT}. Once that's fixed, you might want to check that it can also handle a relative path. An absolute path, without PROJECT_ROOT, worked for me.

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
$ pdm info
PDM version:
  2.6.1
Python Interpreter:
  /home/Tim.Diels/example/foo/.venv/bin/python (3.11)
Project Root:
  /home/Tim.Diels/example/foo
Local Packages:

$ pdm info --env
{
  "implementation_name": "cpython",
  "implementation_version": "3.11.3",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.19.0-41-generic",
  "platform_system": "Linux",
  "platform_version": "#42~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 18 17:40:00 UTC 2",
  "python_full_version": "3.11.3",
  "platform_python_implementation": "CPython",
  "python_version": "3.11",
  "sys_platform": "linux"
}
@hannahdiels-ontoforce hannahdiels-ontoforce added the 🐛 bug Something isn't working label May 15, 2023
@frostming
Copy link
Collaborator

The relative path is backend-specific, setuptools doesn't support it, so we can't do anything here.

@frostming frostming closed this as not planned Won't fix, can't repro, duplicate, stale May 16, 2023
@hannahdiels-ontoforce
Copy link
Author

Alright, thanks for the clarification. I think my workaround will be to split it into a C extension project that doesn't depend on the rest and a pure python one that does. Do you think temporarily changing to setuptools backend only during pdm build with a hook or so would work too?

@frostming
Copy link
Collaborator

@timdiels-ontoforce why not pdm-backend? it also supports calling setuptools for building C extension: https://pdm-backend.fming.dev/hooks/#call-setup-function-to-build-extensions

@hannahdiels-ontoforce
Copy link
Author

Ah great I'll try that, I somehow overlooked that in the docs when searching for "C extension". I then stumbled upon an old discussion on this repo suggesting to use setuptools backend.

@hannahdiels-ontoforce
Copy link
Author

Thanks, that works. I see why I overlooked it, did not realize there's a separate docs site for pdm-backend, only searched the pdm one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants