Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce --lock resolves. #1654

Conversation

jsirois
Copy link
Member

@jsirois jsirois commented Mar 7, 2022

The Pex CLI can now resolve from a lock file created via
pex3 lock create when creating PEX files.

Closes #1583

jsirois added 3 commits March 6, 2022 17:21
For the local case of lock create and then later resolve, it will be
good to retain the (primary) artifacts downloaded for the local
interpreter when creating the lock used to lock in order to save
downloading those artifacts again when later resolving from the lock.

Work towards pex-tool#1583.
Resolving from a lockfile will need this too.
The Pex CLI can now resolve from a lock file created via
`pex3 lock create` when creating PEX files.

Closes pex-tool#1583
@jsirois
Copy link
Member Author

jsirois commented Mar 7, 2022

N.B.: The 1st two commits are under review elsewhere:
e46c06d: #1650
168d64e: #1651

Please just focus on ffd28cd.

Copy link
Collaborator

@benjyw benjyw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really fantastic!

Copy link
Contributor

@Eric-Arellano Eric-Arellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Epic! 🎉

@@ -77,6 +77,7 @@ def _add_resolve_options(cls, parser):
resolver_options.register(
cls._create_resolver_options_group(parser),
include_pex_repository=False,
include_lock=False,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benjyw this is why it's an option to disable registration.



class URLFetcherDownloadManager(DownloadManager):
_BUFFER_SIZE = 65536
Copy link
Contributor

@Eric-Arellano Eric-Arellano Mar 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hadn't seen this number used before but knew you chose it for a reason. Fine if you don't want to burn a tree, but otherwise this might help newbs like me

Suggested change
_BUFFER_SIZE = 65536
_BUFFER_SIZE = 65536 # i.e. max number of 16 bits.

Edit: or probably a better comment would be why this number.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah - not the reason. I can circle back, but, as always, buffer size selection is all about perf and nothing more. The number of bits may tangentially relate to the size of underlying os buffers, but its not the point, the point, hopefully, is you measured on some reasonably diverse set of inputs and found a best-fit. That's what this was.

@jsirois jsirois merged commit 2f2c772 into pex-tool:main Mar 7, 2022
@jsirois jsirois deleted the issues/1583/lockfile-resolve/download-build-install branch March 7, 2022 19:27
@stuhood
Copy link

stuhood commented Mar 11, 2022

Very exciting! Congratulations John.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support resolving using a Pex lock file.
4 participants