Skip to content

Commit

Permalink
Use the bundled pip wheel to perform pip bootstrap
Browse files Browse the repository at this point in the history
Previously the buildpack downloaded a pip wheel from S3 in order
to bootstrap the pip/setuptools/wheel installation.

Newer Python versions (Python 3.4+) bundle a pip wheel as part of
the `ensurepip` module, which we can use instead of having to
perform this download now that we've dropped support for Python 2.

This approach:
- improves performance/reliability of repeat builds, since it saves
  having to download the pip wheel each time
- reduces the toil of performing pip updates, since we no longer need
  to upload new pip versions to the S3 bucket each time
- matches the approach already used in the new Python CNB.

GUS-W-13111316.
  • Loading branch information
edmorley committed Apr 27, 2023
1 parent 72f1600 commit f49089c
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 17 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## Unreleased

- The pip bootstrap step is now performed using the pip wheel bundled with the Python stdlib, rather than one downloaded from S3. ([#1442](https://github.com/heroku/heroku-buildpack-python/pull/1442))

## v231 (2023-04-12)

Expand Down
28 changes: 11 additions & 17 deletions bin/steps/python
Original file line number Diff line number Diff line change
Expand Up @@ -179,25 +179,19 @@ WHEEL_VERSION='0.40.0'

puts-step "Installing pip ${PIP_VERSION}, setuptools ${SETUPTOOLS_VERSION} and wheel ${WHEEL_VERSION}"

# We don't use get-pip.py, since:
# - it uses `--force-reinstall`, which is unnecessary here and slows down repeat builds
# - it means downloading pip twice (once embedded in get-pip.py, and again during
# the install, since get-pip.py can't install the embedded version directly)
# - we would still have to manage several versions of get-pip.py, to support older Pythons.
# Instead, we use the pip wheel to install itself, using the method described here:
# https://github.com/pypa/pip/issues/2351#issuecomment-69994524

PIP_WHEEL_FILENAME="pip-${PIP_VERSION}-py3-none-any.whl"
PIP_WHEEL_URL="${S3_BASE_URL}/common/${PIP_WHEEL_FILENAME}"
PIP_WHEEL="${TMPDIR:-/tmp}/${PIP_WHEEL_FILENAME}"

if ! curl --silent --show-error --fail --retry 3 --retry-connrefused --connect-timeout 10 "${PIP_WHEEL_URL}" -o "$PIP_WHEEL"; then
mcount "failure.python.download-pip"
puts-warn "Failed to download pip"
# Python bundles Pip within its standard library, which we can use to install our chosen
# pip version from PyPI, saving us from having to download the usual pip bootstrap script.
# We have to use a glob since the bundled wheel filename contains the pip version, which
# differs between Python versions.
BUNDLED_PIP_WHEEL="$(compgen -G ".heroku/python/lib/python*/ensurepip/_bundled/pip-*.whl" || true)"

if [[ -z "${BUNDLED_PIP_WHEEL}" ]]; then
mcount "failure.python.locate-bundled-pip-wheel"
puts-warn "Failed to locate the bundled pip wheel"
exit 1
fi

/app/.heroku/python/bin/python "${PIP_WHEEL}/pip" install --quiet --disable-pip-version-check --no-cache-dir \
"${PIP_WHEEL}" "setuptools==${SETUPTOOLS_VERSION}" "wheel==${WHEEL_VERSION}"
/app/.heroku/python/bin/python "${BUNDLED_PIP_WHEEL}/pip" install --quiet --disable-pip-version-check --no-cache-dir \
"pip==${PIP_VERSION}" "setuptools==${SETUPTOOLS_VERSION}" "wheel==${WHEEL_VERSION}"

hash -r

0 comments on commit f49089c

Please sign in to comment.