Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose compression control for Pex zipping operations. #1686

Closed
jsirois opened this issue Mar 23, 2022 · 1 comment · Fixed by #1705
Closed

Expose compression control for Pex zipping operations. #1686

jsirois opened this issue Mar 23, 2022 · 1 comment · Fixed by #1705
Assignees

Comments

@jsirois
Copy link
Member

jsirois commented Mar 23, 2022

The experiments here and here in #1675 prove out that the zip compression level provides a significant swing in tradeoffs. It's probably worth exposing this to let users decide where they want to land in the tradeoff spectrum.

@jsirois
Copy link
Member Author

jsirois commented Apr 4, 2022

For a modest sized PEX the gain is modest, ~25% speed improvement creating the PEX zip, ~no runtime perf impact and ~350% size increase:

$ hyperfine -w1 'python -mpex jupyter -c jupyter -o jupyter.default.pex' 'python -mpex jupyter -c jupyter -o jupyter.no-compress.pex --no-compress'
Benchmark 1: python -mpex jupyter -c jupyter -o jupyter.default.pex
  Time (mean ± σ):      8.695 s ±  0.288 s    [User: 6.429 s, System: 0.253 s]
  Range (min … max):    8.422 s …  9.351 s    10 runs
 
Benchmark 2: python -mpex jupyter -c jupyter -o jupyter.no-compress.pex --no-compress
  Time (mean ± σ):      7.065 s ±  0.109 s    [User: 4.665 s, System: 0.276 s]
  Range (min … max):    6.928 s …  7.254 s    10 runs
 
Summary
  'python -mpex jupyter -c jupyter -o jupyter.no-compress.pex --no-compress' ran
    1.23 ± 0.05 times faster than 'python -mpex jupyter -c jupyter -o jupyter.default.pex'

$ ls -lh jupyter.*.pex
-rwxr-xr-x 1 jsirois jsirois 25M Apr  4 11:05 jupyter.default.pex
-rwxr-xr-x 1 jsirois jsirois 88M Apr  4 11:06 jupyter.no-compress.pex

$ hyperfine -w1 './jupyter.default.pex --version' './jupyter.no-compress.pex --version'
Benchmark 1: ./jupyter.default.pex --version
  Time (mean ± σ):     861.9 ms ±   6.8 ms    [User: 812.1 ms, System: 49.6 ms]
  Range (min … max):   852.6 ms … 873.5 ms    10 runs
 
Benchmark 2: ./jupyter.no-compress.pex --version
  Time (mean ± σ):     871.5 ms ±   9.5 ms    [User: 826.1 ms, System: 45.1 ms]
  Range (min … max):   859.8 ms … 885.8 ms    10 runs
 
Summary
  './jupyter.default.pex --version' ran
    1.01 ± 0.01 times faster than './jupyter.no-compress.pex --version'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant