Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: composite scripts #1117

Merged
merged 1 commit into from
Jun 10, 2022

Conversation

noirbizarre
Copy link
Member

@noirbizarre noirbizarre commented Jun 6, 2022

Pull Request Check List

  • A news fragment is added in news/ describing what is new.
  • Test cases added for changed code.

Describe what you have changed in this PR.

This PR adds a "composite script" feature allowing to run multiple scripts one by one.

A classical use case would be:

[tool.pdm.scripts]
lint = "flake8 mypackage"
test = "pytest -v tests/"
ci = {composite = ["lint", "test"]}

Then pdm run ci will run the lint command and then the test command. Execution stops and returns on first error.

Both env and env_file defined in composite action will override those defined in called tasks.

So with the following:

[tool.pdm.scripts]
echo = {shell = "echo $VAR", env = { VAR = "value"}}
composite = {composite = ["echo"], env = { VAR = "overriden"}}

you will have:

$ pdm run echo
value
$ pdm run composite
overriden

@noirbizarre noirbizarre force-pushed the feature/composite-scripts branch 3 times, most recently from 13e62b5 to 1211dc3 Compare June 6, 2022 02:30
Copy link
Collaborator

@frostming frostming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, this feature is good to have, but since there is still room for improvement and we are developing on v2.0, new features should go there. I will change the base to dev branch if you don't mind.

pdm/cli/commands/run.py Outdated Show resolved Hide resolved
pdm/cli/commands/run.py Outdated Show resolved Hide resolved
pdm/cli/commands/run.py Outdated Show resolved Hide resolved
@frostming frostming changed the base branch from main to dev June 6, 2022 02:41
@frostming frostming added the ⭐ enhancement Improvements for existing features label Jun 6, 2022
@frostming frostming added this to the Release 2.0 milestone Jun 6, 2022
@noirbizarre noirbizarre force-pushed the feature/composite-scripts branch 2 times, most recently from 0ba4d98 to 0189eb3 Compare June 6, 2022 13:42
@noirbizarre
Copy link
Member Author

noirbizarre commented Jun 6, 2022

Given it's now planned for 2.0 and we have time to craft a proper solution, I did the following:

  • rebased on dev branch
  • merged run() and _run_subtask()
  • make run() and _run_task() accept an optional TaskOptions parameter
  • environment variables are cascading

I think I'm going to add:

  • support for passing argument to subtasks
  • more tests on environment handling (at least for cascading)

@noirbizarre noirbizarre force-pushed the feature/composite-scripts branch from 0189eb3 to 484f449 Compare June 7, 2022 21:24
@noirbizarre noirbizarre force-pushed the feature/composite-scripts branch from 484f449 to 37bb6d4 Compare June 7, 2022 21:27
@noirbizarre
Copy link
Member Author

noirbizarre commented Jun 7, 2022

PR updated with:

  • more tests
  • arguments passing to subtasks

I would say scope is OK for a first unless you want or see something else.

Note: previous build failure was a permission denied on github assets under Windows, I don't think this is related to this PR

@codecov-commenter
Copy link

codecov-commenter commented Jun 10, 2022

Codecov Report

Attention: Patch coverage is 94.73684% with 1 line in your changes missing coverage. Please review.

Project coverage is 84.34%. Comparing base (b8878b1) to head (37bb6d4).

Files with missing lines Patch % Lines
pdm/cli/commands/run.py 94.73% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##              dev    #1117   +/-   ##
=======================================
  Coverage   84.33%   84.34%           
=======================================
  Files          79       79           
  Lines        6921     6930    +9     
  Branches     1640     1644    +4     
=======================================
+ Hits         5837     5845    +8     
+ Misses        736      732    -4     
- Partials      348      353    +5     
Flag Coverage Δ
unittests 84.11% <94.73%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@frostming frostming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! LGTM

@frostming frostming merged commit 1c376a8 into pdm-project:dev Jun 10, 2022
@noirbizarre noirbizarre deleted the feature/composite-scripts branch June 10, 2022 06:06
frostming added a commit that referenced this pull request Jun 28, 2022
* feat(core): Use tomllib on Python 3.11 (#1072)

* docs: 📝 Fix typo in `pip install pdm` description (#1061)

* Use tomllib on Python 3.11

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* news

* use a compatibility module

* missed one import

Co-authored-by: t106362512 <[email protected]>
Co-authored-by: hauntsaninja <>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* feat(core): Replace halo, click, and termcolor with rich (#1091)

* feat(core): Use `unearth` as the backend to find and download packages (#1096)

* perf(resolver): Speed up the resolution with lazy find_matches (#1098)

* Improve the output of installing packages

* Use confirm instead of ask

* feat(core): New command: pdm publish (#1107)

* Use rich handler for logging to stderr

* feat(scripts): added composite tasks support (#1117)

* feat(core): Add option to skip hooks (#1127)

* fix(scripts): allow pdm test command to receive path arguments

* feat(hooks): added a `--skip` option to skipp scripts and hooks (#1127)

fix #948

* feat(hooks): use PDM_SKIP_HOOKS environement variable as fallback for skip list

* feat(core): Support setup.py import (#1137)

* Update completion script

* fix(resolution): fix a bug that versions with local part can't be found and installed
Close #1093

* feat(core): forbid editable depenencies in project table (#1140)

* Make the error message more friendly

* doc: improve the docs about dependencies

* doc: add CLI reference doc

* doc: use asciiart as the program description

* chore: remove remaining artifacts from #1127 (#1152)

* Feature: complete lifecycle signals and documentation (#1147)

* feat(hooks): Added pre-publish hook

* refactor(hooks): dynamic signal/hooks listing avoiding double declaration

* feat(hooks): added (pre|post)_script and (pre|post)_run hooks

* doc(hooks): added lifecycle and hooks documentation

* review fix

* fix(tests): add and use the _echo fixture for cross-plateform and concise test echos

* refactor(hooks): automatically register the script handler for all hooks

* feat: Update pdm-pep517 to 1.0 (#1153)

* fix(scripts): merge the Script and Description field from listing (#1151)

* feat: fetch the candidate hashes concurrently (#1154)

* feat: fetch the candidate hashes concurrently

* add news

* Feat/respect-source-order (#1155)

* doc: restructure the docs about project metadata and build configuration

* parse pep 621 metadata to avoid build (#1156)

* feat: Remove the compatible support for pdm legacy metadata (#1157)

* fix(config): use platform standard directories for all PDM directories (#1161)

Fixes #1150

* fix(#1156): only trust parsing result when all are static

* New build configuration table

* chore: added a tox.ini file for easier local testing against all Python versions (#1160)

* feat(CLI): Yarn-like root scripts fallback (#1159)

* feat(hooks): added a post_use hook (#1163)

Co-authored-by: Shantanu <[email protected]>
Co-authored-by: t106362512 <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Axel H <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
⭐ enhancement Improvements for existing features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants