Skip to content

Commit

Permalink
feat: faster ZipCrypto decryption (via Rust), dropping support for Py…
Browse files Browse the repository at this point in the history
…thon 3.6

This decreases the time to decrypt ZipCrypto ZIPs by approximately a factor of
10 (measured using 100MiB of random data zipped).

This uses pyo3 to make Rust code available to Python. However, recent versions
of pyo3 don't support Python 3.6 (and older don't seem to support Python 3.12
onwards), and judging the factor of 10 worth it to drop support. And Python 3.6
has been end-of-life for almost 3 years, so perhaps it's time.
  • Loading branch information
osimuka authored and michalc committed Oct 20, 2024
1 parent 736ab5a commit fffa400
Show file tree
Hide file tree
Showing 9 changed files with 1,055 additions and 70 deletions.
186 changes: 169 additions & 17 deletions .github/workflows/deploy-package-to-pypi.yml
Original file line number Diff line number Diff line change
@@ -1,33 +1,174 @@
name: Deploy package to PyPI

on:
release:
types: [published]
# release:
# types: [published]

push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

jobs:
build:
build-source:
name: Build source package
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

# - name: Update version in pyproject.toml and setup.py from current git tag
# run: |
# sed -i "s/0\\.0\\.0\\.dev0/${GITHUB_REF_NAME}/g" pyproject.toml
# sed -i "s/0\\.0\\.0\\.dev0/${GITHUB_REF_NAME}/g" setup.py

- uses: actions/setup-python@v4
with:
python-version: 3.13

- name: Build package
run: |
pip install build
python -m build --sdist
- uses: actions/upload-artifact@v4
with:
name: source
path: ./dist

build-linux:
name: Build Linux wheels
runs-on: ubuntu-latest
strategy:
matrix:
image:
- "manylinux2014_x86_64"
- "musllinux_1_1_x86_64"
- "manylinux2014_aarch64"
- "musllinux_1_1_aarch64"
folder:
- "cp37-cp37m"
- "cp38-cp38"
- "cp39-cp39"
- "cp310-cp310"
- "cp311-cp311"
- "cp312-cp312"
- "cp313-cp313"

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

# - name: Update version in pyproject.toml and setup.py from current git tag
# run: |
# sed -i "s/0\\.0\\.0\\.dev0/${GITHUB_REF_NAME}/g" pyproject.toml
# sed -i "s/0\\.0\\.0\\.dev0/${GITHUB_REF_NAME}/g" setup.py

- uses: actions/setup-python@v3
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
with:
python-version: 3.11
platforms: arm64

- name: Update version in pyproject.toml from current git tag
- name: Build packages
run: >-
sed -i "s/0\\.0\\.0\\.dev0/${GITHUB_REF/refs\/tags\/v/}/g" pyproject.toml
docker run --rm -v ${{ github.workspace }}:/app quay.io/pypa/${{ matrix.image }} bash -c '
cd /app &&
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y &&
. "$HOME/.cargo/env" &&
/opt/python/${{ matrix.folder }}/bin/python -m build --wheel
auditwheel repair $(ls dist/*.whl) &&
rm dist/*.whl &&
cp wheelhouse/*.whl dist
'
- uses: actions/upload-artifact@v4
with:
name: linux-${{ matrix.image }}-$${{ matrix.folder }}
path: ./dist

build-macos:
name: Build macOS wheels
strategy:
matrix:
os:
- "macos-12"
- "macos-13"
- "macos-14" # ARM
python-version:
- "3.7.1"
- "3.8.10"
- "3.9.13"
- "3.10.11"
- "3.11.9"
- "3.12.6"
- "3.13.0"
exclude:
- python-version: "3.7.1"
os: "macos-14"
runs-on: '${{ matrix.os }}'

- run: |
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: '${{ matrix.python-version }}'

# - name: Update version in pyproject.toml and setup.py from current git tag
# run: |
# sed -i "" "s/0\\.0\\.0\\.dev0/${GITHUB_REF_NAME}/g" pyproject.toml
# sed -i "" "s/0\\.0\\.0\\.dev0/${GITHUB_REF_NAME}/g" setup.py

- name: Build package
run: |
pip install build
python -m build --wheel
- uses: actions/upload-artifact@v4
with:
name: ${{ matrix.os }}-${{ matrix.python-version }}
path: ./dist

build-windows:
name: Build Windows wheels
strategy:
matrix:
os:
- "windows-2019"
python-version:
- "3.7.1"
- "3.8.0"
- "3.9.0"
- "3.10.0"
- "3.11.0"
- "3.12.0"
- "3.13.0"
runs-on: '${{ matrix.os }}'

steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
python-version: '${{ matrix.python-version }}'

# - name: Update version in pyproject.toml and setup.py from current git tag
# run: |
# (Get-Content pyproject.toml).Replace('0.0.0.dev0', $Env:GITHUB_REF_NAME) | Set-Content pyproject.toml
# (Get-Content setup.py).Replace('0.0.0.dev0', $Env:GITHUB_REF_NAME) | Set-Content setup.py

- name: Build package
run: |
pip install build
python -m build
python -m build --wheel
- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: ${{ matrix.os }}-${{ matrix.python-version }}
path: ./dist

deploy:
needs: [build]
needs:
- build-source
- build-linux
- build-macos
- build-windows
environment:
name: pypi
url: https://pypi.org/project/stream-unzip/
Expand All @@ -37,9 +178,20 @@ jobs:
permissions:
id-token: write
steps:
- uses: actions/download-artifact@v3

- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
- uses: actions/download-artifact@v4
with:
packages_dir: artifact/
path: ./dist

# The "merge-multiple" option of download-artifact seems to cause corruption when there are
# multiple files of the same name, which happens because in some different macOS versions
# make the exact same Python package. So we avoid that and do a manual move of packages
# to the top level for upload
- name: Move packages to top level
run: |
find ./dist -mindepth 2 -type f -exec mv -t ./dist -i '{}' +
rm -R -- ./dist/*/
# - name: Publish package distributions to PyPI
# uses: pypa/gh-action-pypi-publish@release/v1
# with:
# packages_dir: ./dist/
2 changes: 0 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,6 @@ jobs:
# If changing how many times tests are run, must also change in codecov.yml
# to ensure test coverage is reported only after all tests have finished
include:
- python-version: "3.6.7"
os: "ubuntu-20.04"
- python-version: "3.7.1"
os: "ubuntu-20.04"
- python-version: "3.7.2"
Expand Down
Loading

0 comments on commit fffa400

Please sign in to comment.