doc: guidelines for PDM usage in CI

[deronnax]

Is your feature request related to a problem? Please describe.

Currently, the documentation gives no instructions for CI: what directories should be cached, or should it even be, should the pip cache be cached, what command you be used in the CI (install vs sync)? Inspiration can be drawn from Poetry and CI builds -- best practices, knowledge share (travisci), and discussion

I am asking because I migrated our pipelines from Pipenv to PDM, and package install time doubled. With pipenv we used to cache the pip cache between jobs, but I suspect PDM is not only reliable on this. Will caching the cache_dir make an impact on installs in CI ? Will it on locking? (we have repos were the pdm.lock is not checked-out).

Thank you.

[frostming]

What about https://pdm-project.org/latest/usage/advanced/#use-pdm-in-continuous-integration

PDM is storing caches at what pdm config cache_dir is pointing to. PDM isn't running pip under the hood, on contrary to what many expect, so it has nothing to do with pip's cache. Besides, you can also cache $working_dir/.venv for each (python version, platform, architecture) combination. All above have been handled by the official pdm-project/setup-pdm GitHub Action.

Feel free to contribute to the docs if there is anything unclear.

[deronnax]

But we use Gitlab CI, so we can't take advantage of the action, otherwise I would just have had and not bothered you 😉
The #use-pdm-in-continuous-integration basically just says to use the GitHub Action, but we are not on GitHub Action.
I will have a quick look at the action's code, and I will try to write down what is relevant to do in a PDM CI job if you can't have the GitHub actions (e.g, do what is needed to cache the PDM cache. Gitlab mandates cached directory be children of the current job run directory, so you have to change the package manager cache location, for one).

[sanmai-NL]

@deronnax I'll write a PR with generic guidance and examples, as we use it on GitLab CI. I'll update the Containerfile example.

One problem I was about to look into is and fix with a PR is #2398, that also runs into that job directory issue.

[deronnax]

OK. I am dying to see it. I will look at it as soon as you create it.
As for me, I don't think we are affected by #2398

[deronnax]

btw, in the multi-stage Dockerfile example, the documentation is still using a __pypackages__, is it something we want to keep?

[pawamoy]

I think it's OK to keep it, though we should provide a venv example too, good catch.

[sanmai-NL]

See #2482 for further discussion please.

[JeyDi]

Hi guys do you have some guidelines on how to build a package using CI and PDM? On Github and Gitlab?
I'm struggling with the tags in particular: I am trying to dynamically assign my tag from the CD to the python package I want to build with PMD build, do you have any ideas or examples?

[sanmai-NL]

Hi guys do you have some guidelines on how to build a package using CI and PDM? On Github and Gitlab? I'm struggling with the tags in particular: I am trying to dynamically assign my tag from the CD to the python package I want to build with PMD build, do you have any ideas or examples?

Please ask this as a Discussions question.