Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support PEP 735 dependency groups #3230

Merged
merged 2 commits into from
Oct 29, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/usage/advanced.md
Original file line number Diff line number Diff line change
@@ -186,7 +186,7 @@ With PDM, you can have multiple sub-packages within a single project, each with
`project/pyproject.toml`:

```toml
[tool.pdm.dev-dependencies]
[dependency-groups]
dev = [
"-e file:///${PROJECT_ROOT}/packages/foo-core",
"-e file:///${PROJECT_ROOT}/packages/foo-cli",
@@ -244,7 +244,7 @@ This hook wraps the command `pdm lock --check` along with any valid argument. It

### Sync current working set with `pdm.lock`

This hook wraps the command `pdm sync` along with any valid argument. It can be used as a hook to ensure that your current working set is synced with `pdm.lock` whenever you checkout or merge a branch. Add *keyring* to `additional_dependencies` if you want to use your systems credential store.
This hook wraps the command `pdm sync` along with any valid argument. It can be used as a hook to ensure that your current working set is synced with `pdm.lock` whenever you checkout or merge a branch. Add _keyring_ to `additional_dependencies` if you want to use your systems credential store.

```yaml
- repo: https://github.com/pdm-project/pdm
12 changes: 6 additions & 6 deletions docs/usage/dependency.md
Original file line number Diff line number Diff line change
@@ -137,15 +137,15 @@ pdm add -dG test pytest
This will result in a pyproject.toml as following:

```toml
[tool.pdm.dev-dependencies]
[dependency-groups]
test = ["pytest"]
```

You can have several groups of development only dependencies. Unlike `optional-dependencies`, they won't appear in the package distribution metadata such as `PKG-INFO` or `METADATA`.
The package index won't be aware of these dependencies. The schema is similar to that of `optional-dependencies`, except that it is in `tool.pdm` table.

```toml
[tool.pdm.dev-dependencies]
[dependency-groups]
lint = [
"flake8",
"black"
@@ -154,10 +154,10 @@ test = ["pytest", "pytest-cov"]
doc = ["mkdocs"]
```

For backward-compatibility, if only `-d` or `--dev` is specified, dependencies will go to `dev` group under `[tool.pdm.dev-dependencies]` by default.
For backward-compatibility, if only `-d` or `--dev` is specified, dependencies will go to `dev` group under `[dependency-groups]` by default.

!!! NOTE
The same group name MUST NOT appear in both `[tool.pdm.dev-dependencies]` and `[project.optional-dependencies]`.
The same group name MUST NOT appear in both `[dependency-groups]` and `[project.optional-dependencies]`.

### Editable dependencies

@@ -258,7 +258,7 @@ To remove existing dependencies from project file and the library directory:
pdm remove requests
# Remove h11 from the 'web' group of optional-dependencies
pdm remove -G web h11
# Remove pytest-cov from the `test` group of dev-dependencies
# Remove pytest-cov from the `test` group of dependency-groups
pdm remove -dG test pytest-cov
```

@@ -290,7 +290,7 @@ dependencies = ["requests"]
extra1 = ["flask"]
extra2 = ["django"]

[tool.pdm.dev-dependencies] # This is dev dependencies
[dependency-groups] # This is dev dependencies
dev1 = ["pytest"]
dev2 = ["mkdocs"]
```
7 changes: 3 additions & 4 deletions docs/usage/lockfile.md
Original file line number Diff line number Diff line change
@@ -25,8 +25,7 @@ There are a few similar commands to do this job with slight differences:

- `--clean`: will remove packages no longer in the lockfile
- `--clean-unselected` (or `--only-keep`): more thorough version of `--clean` that will also remove packages not in the groups specified by the `-G`, `-d`, and `--prod` options.
Note: by default, `pdm sync` selects all groups from the lockfile, so `--clean-unselected` is identical to `--clean` unless `-G`, `-d`, and `--prod` are used.

Note: by default, `pdm sync` selects all groups from the lockfile, so `--clean-unselected` is identical to `--clean` unless `-G`, `-d`, and `--prod` are used.

## Hashes in the lock file

@@ -44,7 +43,7 @@ If you want to refresh the lock file without changing the dependencies, you can
pdm lock --refresh
```

This command also refreshes *all* file hashes recorded in the lock file.
This command also refreshes _all_ file hashes recorded in the lock file.

## Specify another lock file to use

@@ -65,7 +64,7 @@ For a realistic example, your project depends on a release version of `werkzeug`
requires-python = ">=3.7"
dependencies = ["werkzeug"]

[tool.pdm.dev-dependencies]
[dependency-groups]
dev = ["werkzeug @ file:///${PROJECT_ROOT}/dev/werkzeug"]
```

1 change: 1 addition & 0 deletions news/3230.feature.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Support dependency groups as standardized by [PEP 735](https://peps.python.org/pep-0735/). By default, dev dependencies will be written to `[dependency-groups]` table.
6 changes: 1 addition & 5 deletions src/pdm/cli/commands/add.py
Original file line number Diff line number Diff line change
@@ -122,11 +122,7 @@ def do_add(
if project.enable_write_lockfile:
project.core.ui.info(f"Adding group [success]{group}[/] to lockfile")
lock_groups.append(group)
if (
group == "default"
or not selection.dev
and group not in project.pyproject.settings.get("dev-dependencies", {})
):
if group == "default" or not selection.dev and group not in project.pyproject.dev_dependencies:
if editables:
raise PdmUsageError("Cannot add editables to the default or optional dependency group")
for r in [parse_requirement(line, True) for line in editables] + [parse_requirement(line) for line in packages]:
5 changes: 2 additions & 3 deletions src/pdm/cli/commands/remove.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from __future__ import annotations

import argparse
from typing import TYPE_CHECKING, cast
from typing import TYPE_CHECKING

from pdm.cli.commands.base import BaseCommand
from pdm.cli.filters import GroupSelection
@@ -82,7 +82,6 @@ def do_remove(
hooks: HookManager | None = None,
) -> None:
"""Remove packages from working set and pyproject.toml"""
from tomlkit.items import Array

from pdm.cli.actions import do_lock, do_sync
from pdm.cli.utils import check_project_file
@@ -111,7 +110,7 @@ def do_remove(
for i in matched_indexes:
del deps[i]
tracked_names.add(normalize_name(name))
setter(cast(Array, deps).multiline(True))
setter(deps)

if not dry_run:
project.pyproject.write()
2 changes: 1 addition & 1 deletion src/pdm/cli/filters.py
Original file line number Diff line number Diff line change
@@ -79,7 +79,7 @@ def _translated_groups(self) -> list[str]:
dev = True
project = self.project
optional_groups = set(project.pyproject.metadata.get("optional-dependencies", {}))
dev_groups = set(project.pyproject.settings.get("dev-dependencies", {}))
dev_groups = set(project.pyproject.dev_dependencies)
groups_set = set(groups)
if groups_set & dev_groups:
if not dev:
4 changes: 2 additions & 2 deletions src/pdm/cli/options.py
Original file line number Diff line number Diff line change
@@ -256,7 +256,7 @@ def no_isolation_option(
metavar="GROUP",
action=split_lists(","),
help="Select group of optional-dependencies separated by comma "
"or dev-dependencies (with `-d`). Can be supplied multiple times, "
"or dependency-groups (with `-d`). Can be supplied multiple times, "
'use ":all" to include all groups under the same species.',
default=[],
)
@@ -265,7 +265,7 @@ def no_isolation_option(
dest="excluded_groups",
metavar="",
action=split_lists(","),
help="Exclude groups of optional-dependencies or dev-dependencies",
help="Exclude groups of optional-dependencies or dependency-groups",
default=[],
)
groups_group.add_argument(
3 changes: 0 additions & 3 deletions src/pdm/installers/core.py
Original file line number Diff line number Diff line change
@@ -5,7 +5,6 @@
from pdm.environments import BaseEnvironment
from pdm.models.requirements import Requirement
from pdm.resolver.reporters import LockReporter
from pdm.resolver.resolvelib import RLResolver


def install_requirements(
@@ -29,8 +28,6 @@ def install_requirements(
keep_self=True,
reporter=reporter,
)
if isinstance(resolver, RLResolver):
resolver.provider.repository.find_dependencies_from_local = False
resolved = resolver.resolve().packages
syncer = environment.project.get_synchronizer(quiet=True)(
environment,
22 changes: 0 additions & 22 deletions src/pdm/models/repositories/base.py
Original file line number Diff line number Diff line change
@@ -72,8 +72,6 @@ def __init__(
self._candidate_info_cache = environment.project.make_candidate_info_cache()
self._hash_cache = environment.project.make_hash_cache()
self.has_warnings = False
self.collected_groups: set[str] = set()
self.find_dependencies_from_local = True
if ignore_compatibility is not NotSet: # pragma: no cover
deprecation_warning(
"The ignore_compatibility argument is deprecated and will be removed in the future. "
@@ -284,26 +282,6 @@ def _get_dependencies_from_metadata(self, candidate: Candidate) -> CandidateMeta
summary = prepared.metadata.metadata.get("Summary", "")
return CandidateMetadata(deps, requires_python, summary)

def _get_dependencies_from_local_package(self, candidate: Candidate) -> CandidateMetadata:
"""Adds the local package as a candidate only if the candidate
name is the same as the local package."""
project = self.environment.project
if not project.is_distribution or candidate.name != project.name:
raise CandidateInfoNotFound(candidate) from None

reqs: list[Requirement] = []
if candidate.req.extras is not None:
all_groups = set(project.iter_groups())
for extra in candidate.req.extras:
if extra in all_groups:
reqs.extend(project.get_dependencies(extra))
self.collected_groups.add(extra)
return CandidateMetadata(
reqs,
str(self.environment.python_requires),
project.pyproject.metadata.get("description", "UNKNOWN"),
)

def get_hashes(self, candidate: Candidate) -> list[FileHash]:
"""Get hashes of all possible installable candidates
of a given package version.
5 changes: 1 addition & 4 deletions src/pdm/models/repositories/lock.py
Original file line number Diff line number Diff line change
@@ -136,10 +136,7 @@ def _get_dependencies_from_lockfile(self, candidate: Candidate) -> CandidateMeta
return CandidateMetadata(deps, candidate.requires_python, entry.summary)

def dependency_generators(self) -> Iterable[Callable[[Candidate], CandidateMetadata]]:
return (
self._get_dependencies_from_local_package,
self._get_dependencies_from_lockfile,
)
return (self._get_dependencies_from_lockfile,)

def _matching_entries(self, requirement: Requirement) -> Iterable[Package]:
for key, entry in self.packages.items():
2 changes: 0 additions & 2 deletions src/pdm/models/repositories/pypi.py
Original file line number Diff line number Diff line change
@@ -51,8 +51,6 @@ def _get_dependencies_from_json(self, candidate: Candidate) -> CandidateMetadata

def dependency_generators(self) -> Iterable[Callable[[Candidate], CandidateMetadata]]:
yield self._get_dependencies_from_cache
if self.find_dependencies_from_local:
yield self._get_dependencies_from_local_package
if self.environment.project.config["pypi.json_api"]:
yield self._get_dependencies_from_json
yield self._get_dependencies_from_metadata
Loading