-
-
Notifications
You must be signed in to change notification settings - Fork 646
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce # of resolves when packaging pex_binary
and python_awslambda
targets
#12343
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Rust tests and lints will be skipped. Delete if not intended. [ci skip-rust] # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]
I think we may actually want to cherry-pick this to 2.6.x? It looks like this is resulting in more work than necessary, like we were setting up multiple repository PEXes (which is how the code reads to me). Compare before:
After:
|
Eric-Arellano
changed the title
[internal] Remove
Reduce # of resolves when packaging Jul 14, 2021
TwoStepPex
abstractionpex_binary
and python_awslambda
targets
stuhood
approved these changes
Jul 14, 2021
Eric-Arellano
added a commit
to Eric-Arellano/pants
that referenced
this pull request
Jul 14, 2021
We already use repository PEXes when `[python-setup].resolve_all_constraints` or experimental lockfile are set, thanks to code in `pex_from_targets.py`. Those benefit from using the same repository PEX as the rest of the codebase, e.g. when using MyPy or Pytest. However, we were ignoring this repository pex with `python_awslambda` and `pex_binary` and resolving more granular repository PEXes after having already resolved the global one. Iiuc, those more granular ones are not coming from the global one. This is wasted work. While a `TwoStepPex` could be useful in those two callsites if you are not using constraints/lockfiles, we strongly encourage using lockfiles. Removing `TwoStepPex` allows us to remove complexity in a hairy part of the codebase, especially as we add new complexity for lockfile support. [ci skip-rust] [ci skip-build-wheels]
Merged
wisechengyi
added a commit
that referenced
this pull request
Jul 17, 2021
### Internal * [internal] Manually fix Black lockfile to handle interpreter constraints ([#12366](#12366)) * Revert "Prefix the entire setup.py chroot. (#12359)" ([#12370](#12370)) * [internal] Cache native client binary in CI ([#12355](#12355)) * Prefix the entire setup.py chroot. ([#12359](#12359)) * [internal] Fix AWS CLI breaking due to Python 2 usage ([#12364](#12364)) * [Internal] Add `git_url()` helper to `docutil.py` ([#12352](#12352)) * [Internal] Refactor how `PythonToolBase` exposes requirements and interpreter constraints ([#12356](#12356)) * [internal] Add experimental per-tool lockfiles with Docformatter as an example ([#12346](#12346)) * Remove Pants's dpeendency on `requests` ([#12348](#12348)) * [internal] Remove `TwoStepPex` abstraction ([#12343](#12343)) * Add psycopg2-binary to default module mapping ([#12339](#12339)) * [internal] Upgrade toolchain pants plugin to 0.13.1 ([#12338](#12338)) * Add an API to coarsen/partition Targets by their cycles ([#12251](#12251)) * Prepare 2.5.1 ([#12329](#12329)) * Bootstrap fewer JVM versions in Coursier/javac tests to hopefully reduce CI flakiness ([#12325](#12325)) * Native client respects `--concurrent`. ([#12324](#12324)) * Add client lib tests. ([#12322](#12322)) * Special case enum option parse failures. ([#12281](#12281))
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We already use repository PEXes when
[python-setup].resolve_all_constraints
or experimental lockfile are set, thanks to code inpex_from_targets.py
. Those benefit from using the same repository PEX as the rest of the codebase, e.g. when using MyPy or Pytest.However, we were ignoring this repository pex with
python_awslambda
andpex_binary
and resolving more granular repository PEXes after having already resolved the global one. Iiuc, those more granular ones are not coming from the global one. This is wasted work.While a
TwoStepPex
could be useful in those two callsites if you are not using constraints/lockfiles, we strongly encourage using lockfiles.Removing
TwoStepPex
allows us to remove complexity in a hairy part of the codebase, especially as we add new complexity for lockfile support.[ci skip-rust]
[ci skip-build-wheels]