Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pipx: fix state=latest w/ install_deps=true #6303

Merged

Conversation

darkrain42
Copy link
Contributor

@darkrain42 darkrain42 commented Apr 8, 2023

SUMMARY
  • Document the minimum version of pipx (0.16.2.1) that works with both pipx and pipx_info modules. I'm not changing the status quo; just documenting a preexisting limit since the pipx module was first introduced in f1807d3, where (as today) the module relies on "pipx list --json --include-injected". The same limitation applies to the pipx_info module since its introduction in 7ffe653
  • Fix a bug that prevents state=latest and install_deps=true from working, because the pipx upgrade command doesn't accept a --include-deps parameter, and hasn't since pipx 0.15. (It's safe to remove this since, per above, the module requires pipx 0.16.2.1).
ISSUE TYPE
  • Bugfix Pull Request
  • Docs Pull Request
COMPONENT NAME

pipx

ADDITIONAL INFORMATION

The failure that occurs with install_deps=true and state=latest looks like this:

- community.general.pipx:
    state: latest
    name: tox
    install_deps: true
fatal: [testhost]: FAILED! => {"changed": false, "cmd": "/usr/bin/python3.10 -m pipx upgrade --include-deps tox", "msg": "usage: /usr/bin/python3.10 -m pipx [-h] [--version]\n                                   {install,uninject,inject,upgrade,upgrade-all,uninstall,uninstall-all,reinstall,reinstall-all,list,run,runpip,ensurepath,environment,completions}\n                                   ...\n/usr/bin/python3.10 -m pipx: error: unrecognized arguments: --include-deps", "rc": 2, "stderr": "usage: /usr/bin/python3.10 -m pipx [-h] [--version]\n                                   {install,uninject,inject,upgrade,upgrade-all,uninstall,uninstall-all,reinstall,reinstall-all,list,run,runpip,ensurepath,environment,completions}\n                                   ...\n/usr/bin/python3.10 -m pipx: error: unrecognized arguments: --include-deps\n", "stderr_lines": ["usage: /usr/bin/python3.10 -m pipx [-h] [--version]", "                                   {install,uninject,inject,upgrade,upgrade-all,uninstall,uninstall-all,reinstall,reinstall-all,list,run,runpip,ensurepath,environment,completions}", "                                   ...", "/usr/bin/python3.10 -m pipx: error: unrecognized arguments: --include-deps"], "stdout": "", "stdout_lines": []}

Since their introduction, these modules rely on 'pipx list --json' to
return machine-readable output about installed pipx applications. That
functionality was introduced in 0.16.2, along with a critical bug fix
(invalid json) in 0.16.2.1.
"pipx upgrade" stopped supporting the "--include-deps" option
("install_deps" in the ansible module) in pipx 0.15
(https://pypa.github.io/pipx/changelog/#01500).

The lack of support causes the pipx module to fail if attempting to use
state=latest with install_deps, since the parameter is passed to both
pipx install (fine) and pipx upgrade (fails).
@ansibullbot
Copy link
Collaborator

cc @russoz
click here for bot help

@ansibullbot ansibullbot added WIP Work in progress bug This issue/PR relates to a bug integration tests/integration language module module module_utils module_utils packaging plugins plugin (any type) tests tests labels Apr 8, 2023
@darkrain42 darkrain42 marked this pull request as ready for review April 8, 2023 19:06
@ansibullbot ansibullbot removed the WIP Work in progress label Apr 8, 2023
@github-actions
Copy link

github-actions bot commented Apr 8, 2023

Docs Build 📝

Thank you for contribution!✨

The docsite for this PR is available for download as an artifact from this run:
https://github.com/ansible-collections/community.general/actions/runs/4651270734

File changes:

  • M collections/community/general/pipx_info_module.html
  • M collections/community/general/pipx_module.html
Click to see the diff comparison.

NOTE: only file modifications are shown here. New and deleted files are excluded.
See the file list and check the published docs to see those files.

diff --git a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/pipx_info_module.html b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/pipx_info_module.html
index 824f575..cc359db 100644
--- a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/pipx_info_module.html
+++ b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/pipx_info_module.html
@@ -264,6 +264,7 @@
 <li><p>This module does not install the <code class="docutils literal notranslate"><span class="pre">pipx</span></code> python package, however that can be easily done with the module <a class="reference external" href="https://docs.ansible.com/ansible/devel/collections/ansible/builtin/pip_module.html#ansible-collections-ansible-builtin-pip-module" title="(in Ansible vdevel)"><span class="xref std std-ref">ansible.builtin.pip</span></a>.</p></li>
 <li><p>This module does not require <code class="docutils literal notranslate"><span class="pre">pipx</span></code> to be in the shell <code class="docutils literal notranslate"><span class="pre">PATH</span></code>, but it must be loadable by Python as a module.</p></li>
 <li><p>This module will honor <code class="docutils literal notranslate"><span class="pre">pipx</span></code> environment variables such as but not limited to <code class="docutils literal notranslate"><span class="pre">PIPX_HOME</span></code> and <code class="docutils literal notranslate"><span class="pre">PIPX_BIN_DIR</span></code> passed using the <a class="reference external" href="https://docs.ansible.com/ansible/devel/playbook_guide/playbooks_environment.html#playbooks-environment" title="(in Ansible vdevel)"><span class="xref std std-ref">environment Ansible keyword</span></a>.</p></li>
+<li><p>This module requires <code class="docutils literal notranslate"><span class="pre">pipx</span></code> version 0.16.2.1 or above.</p></li>
 <li><p>Please note that <code class="docutils literal notranslate"><span class="pre">pipx</span></code> requires Python 3.6 or above.</p></li>
 <li><p>See also the <code class="docutils literal notranslate"><span class="pre">pipx</span></code> documentation at <a class="reference external" href="https://pypa.github.io/pipx/">https://pypa.github.io/pipx/</a>.</p></li>
 </ul>
diff --git a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/pipx_module.html b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/pipx_module.html
index 27f06b7..a8ebb6f 100644
--- a/home/runner/work/community.general/community.general/docsbuild/base/collections/community/general/pipx_module.html
+++ b/home/runner/work/community.general/community.general/docsbuild/head/collections/community/general/pipx_module.html
@@ -252,7 +252,7 @@
 <a class="ansibleOptionLink" href="#parameter-install_deps" title="Permalink to this option"></a><p class="ansible-option-type-line"><span class="ansible-option-type">boolean</span></p>
 </div></td>
 <td><div class="ansible-option-cell"><p>Include applications of dependent packages.</p>
-<p>Only used when <em>state=install</em>, <em>state=latest</em>, <em>state=upgrade</em>, or <em>state=inject</em>.</p>
+<p>Only used when <em>state=install</em>, <em>state=latest</em>, or <em>state=inject</em>.</p>
 <p class="ansible-option-line"><span class="ansible-option-choices">Choices:</span></p>
 <ul class="simple">
 <li><p><code class="ansible-option-default-bold docutils literal notranslate"><span class="pre">false</span></code> <span class="ansible-option-choices-default-mark">← (default)</span></p></li>
@@ -354,6 +354,7 @@
 <li><p>This module does not install the <code class="docutils literal notranslate"><span class="pre">pipx</span></code> python package, however that can be easily done with the module <a class="reference external" href="https://docs.ansible.com/ansible/devel/collections/ansible/builtin/pip_module.html#ansible-collections-ansible-builtin-pip-module" title="(in Ansible vdevel)"><span class="xref std std-ref">ansible.builtin.pip</span></a>.</p></li>
 <li><p>This module does not require <code class="docutils literal notranslate"><span class="pre">pipx</span></code> to be in the shell <code class="docutils literal notranslate"><span class="pre">PATH</span></code>, but it must be loadable by Python as a module.</p></li>
 <li><p>This module will honor <code class="docutils literal notranslate"><span class="pre">pipx</span></code> environment variables such as but not limited to <code class="docutils literal notranslate"><span class="pre">PIPX_HOME</span></code> and <code class="docutils literal notranslate"><span class="pre">PIPX_BIN_DIR</span></code> passed using the <a class="reference external" href="https://docs.ansible.com/ansible/devel/playbook_guide/playbooks_environment.html#playbooks-environment" title="(in Ansible vdevel)"><span class="xref std std-ref">environment Ansible keyword</span></a>.</p></li>
+<li><p>This module requires <code class="docutils literal notranslate"><span class="pre">pipx</span></code> version 0.16.2.1 or above.</p></li>
 <li><p>Please note that <code class="docutils literal notranslate"><span class="pre">pipx</span></code> requires Python 3.6 or above.</p></li>
 <li><p>This first implementation does not verify whether a specified version constraint has been installed or not. Hence, when using version operators, <code class="docutils literal notranslate"><span class="pre">pipx</span></code> module will always try to execute the operation, even when the application was previously installed. This feature will be added in the future.</p></li>
 <li><p>See also the <code class="docutils literal notranslate"><span class="pre">pipx</span></code> documentation at <a class="reference external" href="https://pypa.github.io/pipx/">https://pypa.github.io/pipx/</a>.</p></li>

@russoz
Copy link
Collaborator

russoz commented Apr 9, 2023

Hi @darkrain42 Thanks for your contribution! :-)

Please submit the bug fix in a PR of its own, so it can be backported to previous versions. The rest of the PR looks good on a first glance but I shall review it more thoroughly in the next couple of days.

@felixfontein felixfontein added check-before-release PR will be looked at again shortly before release and merged if possible. backport-6 labels Apr 9, 2023
@darkrain42 darkrain42 force-pushed the pipx-fixes-and-improvements branch from 3fc1983 to be9465d Compare April 9, 2023 15:28
@darkrain42 darkrain42 changed the title pipx: fix state=latest w/ install_deps=true, and add support for system_site_packages pipx: fix state=latest w/ install_deps=true Apr 9, 2023
@darkrain42
Copy link
Contributor Author

darkrain42 commented Apr 9, 2023

Created #6308 for the system_site_packages feature and left this with the bug fix (and the min version of pipx that is required to use the pipx + pipx_info modules)

@felixfontein
Copy link
Collaborator

@russoz any more comments?

Copy link
Collaborator

@russoz russoz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nop, it LGTM

@ansibullbot ansibullbot added the stale_ci CI is older than 7 days, rerun before merging label Apr 19, 2023
@felixfontein felixfontein removed the check-before-release PR will be looked at again shortly before release and merged if possible. label Apr 20, 2023
@felixfontein felixfontein merged commit 996fc8c into ansible-collections:main Apr 20, 2023
@patchback
Copy link

patchback bot commented Apr 20, 2023

Backport to stable-5: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 996fc8c on top of patchback/backports/stable-5/996fc8c18e43dcadf32c06ef6179d8b1d0c3bae5/pr-6303

Backporting merged PR #6303 into main

  1. Ensure you have a local repo clone of your fork. Unless you cloned it
    from the upstream, this would be your origin remote.
  2. Make sure you have an upstream repo added as a remote too. In these
    instructions you'll refer to it by the name upstream. If you don't
    have it, here's how you can add it:
    $ git remote add upstream https://github.com/ansible-collections/community.general.git
  3. Ensure you have the latest copy of upstream and prepare a branch
    that will hold the backported code:
    $ git fetch upstream
    $ git checkout -b patchback/backports/stable-5/996fc8c18e43dcadf32c06ef6179d8b1d0c3bae5/pr-6303 upstream/stable-5
  4. Now, cherry-pick PR pipx: fix state=latest w/ install_deps=true #6303 contents into that branch:
    $ git cherry-pick -x 996fc8c18e43dcadf32c06ef6179d8b1d0c3bae5
    If it'll yell at you with something like fatal: Commit 996fc8c18e43dcadf32c06ef6179d8b1d0c3bae5 is a merge but no -m option was given., add -m 1 as follows instead:
    $ git cherry-pick -m1 -x 996fc8c18e43dcadf32c06ef6179d8b1d0c3bae5
  5. At this point, you'll probably encounter some merge conflicts. You must
    resolve them in to preserve the patch from PR pipx: fix state=latest w/ install_deps=true #6303 as close to the
    original as possible.
  6. Push this branch to your fork on GitHub:
    $ git push origin patchback/backports/stable-5/996fc8c18e43dcadf32c06ef6179d8b1d0c3bae5/pr-6303
  7. Create a PR, ensure that the CI is green. If it's not — update it so that
    the tests and any other checks pass. This is it!
    Now relax and wait for the maintainers to process your pull request
    when they have some cycles to do reviews. Don't worry — they'll tell you if
    any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@patchback
Copy link

patchback bot commented Apr 20, 2023

Backport to stable-6: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-6/996fc8c18e43dcadf32c06ef6179d8b1d0c3bae5/pr-6303

Backported as #6377

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

@felixfontein
Copy link
Collaborator

@darkrain42 thanks for your contribution!
@russoz thanks for reviewing!

patchback bot pushed a commit that referenced this pull request Apr 20, 2023
* pipx and pipx_info: Document that modules require pipx 0.16.2.1 or above

Since their introduction, these modules rely on 'pipx list --json' to
return machine-readable output about installed pipx applications. That
functionality was introduced in 0.16.2, along with a critical bug fix
(invalid json) in 0.16.2.1.

* pipx: fix state=latest with install_deps=true

"pipx upgrade" stopped supporting the "--include-deps" option
("install_deps" in the ansible module) in pipx 0.15
(https://pypa.github.io/pipx/changelog/#01500).

The lack of support causes the pipx module to fail if attempting to use
state=latest with install_deps, since the parameter is passed to both
pipx install (fine) and pipx upgrade (fails).

* Add changelog fragment

(cherry picked from commit 996fc8c)
felixfontein pushed a commit that referenced this pull request Apr 20, 2023
…all_deps=true (#6377)

pipx: fix state=latest w/ install_deps=true (#6303)

* pipx and pipx_info: Document that modules require pipx 0.16.2.1 or above

Since their introduction, these modules rely on 'pipx list --json' to
return machine-readable output about installed pipx applications. That
functionality was introduced in 0.16.2, along with a critical bug fix
(invalid json) in 0.16.2.1.

* pipx: fix state=latest with install_deps=true

"pipx upgrade" stopped supporting the "--include-deps" option
("install_deps" in the ansible module) in pipx 0.15
(https://pypa.github.io/pipx/changelog/#01500).

The lack of support causes the pipx module to fail if attempting to use
state=latest with install_deps, since the parameter is passed to both
pipx install (fine) and pipx upgrade (fails).

* Add changelog fragment

(cherry picked from commit 996fc8c)

Co-authored-by: Paul Aurich <[email protected]>
@darkrain42 darkrain42 deleted the pipx-fixes-and-improvements branch April 22, 2023 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue/PR relates to a bug integration tests/integration language module_utils module_utils module module packaging plugins plugin (any type) stale_ci CI is older than 7 days, rerun before merging tests tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants