packages parameter for the pre_install and post_install hooks is given as an exhausted generator

[PKilcommons]

• I have searched the issue tracker and believe that this is not a duplicate.

Ran into a regression while migrating a plugin to 2.19 where the pre_install and post_install hooks' packages parameter is always an exhausted generator. Looks like it's due to the recent changes to the sync internals in cdb025c. Full details in the "Root Cause" section.

Steps to reproduce

Plugin

# repro-plugin/src/repro_plugin/__init__.py

from __future__ import annotations

from typing import TYPE_CHECKING, reveal_type

from pdm.models.repositories import Package
from pdm.signals import post_install, pre_install

if TYPE_CHECKING:
    from pdm.core import Core
    from pdm.project import Project


def _preinstall_hook(project: Project, packages: list[Package], dry_run: bool, **_):
    if dry_run:
        return
    
    print("pre_install:")
    reveal_type(packages)
    print("Package count:", len(list(packages)))


def _postinstall_hook(project: Project, packages: list[Package], dry_run: bool, **_):
    if dry_run:
        return

    print("post_install:")
    reveal_type(packages)
    print("Package count:", len(list(packages)))


def main(_: Core):
    pre_install.connect(_preinstall_hook)
    post_install.connect(_postinstall_hook)

# ./repro-plugin/pyproject.toml
[project]
name = "repro-plugin"
version = "0.0.0"
description = "pre_install and post_install hook reproduction"
dependencies = [
    "pdm==2.19.1",
]
requires-python = "==3.12.*"
readme = "README.md"
license = {text = "MIT"}

[project.entry-points.pdm]
repro = "repro_plugin:main"

Example project

# ./pyproject.toml
[project]
name = "example-project"
version = "0.0.0"
description = "Example project using the `pdm-repro` plugin"
dependencies = [
    "requests>=2.32.3",
]
requires-python = "==3.12.*"
readme = "README.md"
license = {text = "MIT"}

[tool.pdm]
plugins = [
    "-e file:///${PROJECT_ROOT}/repro-plugin"
]

[tool.pdm.dev-dependencies]
dev = [
    "-e repro-plugin @ file:///${PROJECT_ROOT}/repro-plugin",
]

$ pdm install
pre_install:
Runtime type is 'generator'
Package count: 0
Synchronizing working set with resolved packages: 0 to add, 1 to update, 0 to remove

  ✔ Update repro-plugin 0.0.0 -> 0.0.0 successful
  ✔ Update example-project 0.0.0 -> 0.0.0 successful

  0:00:01 🎉 All complete! 1/1
post_install:
Runtime type is 'generator'
Package count: 0

Actual behavior

The packages parameter is exhausted and will produce no elements when iterated over.

Root Cause

The packages variable, which replaced the previous candidates dict, is now a generator:

pdm/src/pdm/cli/actions.py

Line 284 in 16933a1

packages = resolve_from_lockfile(project, requirements, groups=list(selection))

... which is passed to the BaseSynchronizer constructor then iterated over and exhausted.

pdm/src/pdm/cli/actions.py

Line 296 in 16933a1

packages=packages,

pdm/src/pdm/installers/base.py

Line 63 in 16933a1

self.requested_candidates = {entry.candidate.identify(): entry.candidate for entry in packages}

The now exhausted generator object is then passed to the hooks:

pdm/src/pdm/cli/actions.py

Lines 300 to 302 in 16933a1

	hooks.try_emit("pre_install", packages=packages, dry_run=dry_run)
	synchronizer.synchronize()
	hooks.try_emit("post_install", packages=packages, dry_run=dry_run)

... where trying to iterator over it (packages) in the hooks will yield zero elements.

Also worth pointing out that self.packages is assigned to this same exhausted generator object. Maybe this is intentional in this case? I'm not too familiar with pdm's internals.

pdm/src/pdm/installers/base.py

Line 81 in 16933a1

self.packages = packages

Expected behavior

The pre_install and post_install hooks should be passed a list[Package] as documented or some other form of non-empty MutableSequence, such that the previous behaviour of being able to mutate the candidates dict inflight is preserved.

A straight forward fix would be to consume the generator into a list before passing it to the BaseSynchronizer constructor and hooks.

-packages = resolve_from_lockfile(project, requirements, groups=list(selection))
+packages = list(resolve_from_lockfile(project, requirements, groups=list(selection)))

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
PDM version:
  2.19.1
Python Interpreter:
  /home/pmk/projects/pdm-repro/.venv/bin/python (3.12)
Project Root:
  /home/pmk/projects/pdm-repro
Local Packages:
  
{
  "implementation_name": "cpython",
  "implementation_version": "3.12.6",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.4.0-190-generic",
  "platform_system": "Linux",
  "platform_version": "#210-Ubuntu SMP Fri Jul 5 17:03:38 UTC 2024",
  "python_full_version": "3.12.6",
  "platform_python_implementation": "CPython",
  "python_version": "3.12",
  "sys_platform": "linux"
}

[frostming]

good catch. would you like to send a pr?

[PKilcommons]

Sure. I'll see if I can find some time tonight or tomorrow morning to submit one.