Skip to content

Commit

Permalink
Merge branch 'main' into issue/11718
Browse files Browse the repository at this point in the history
  • Loading branch information
stephenfin authored Jan 23, 2023
2 parents c46537f + e69e265 commit 8278b19
Show file tree
Hide file tree
Showing 17 changed files with 454 additions and 24 deletions.
16 changes: 8 additions & 8 deletions docs/html/development/architecture/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ Things pip does:
backwards compatibility reasons. But thing with setuptools:
has a ``setup.py`` file that it invokes to …… get info?

2. Decides where to install stuff. Once the package is built, resulting
artifact is then installed into system in appropriate place. :pep:`517`
defines interface between build backend & installer.
2. Decides where to install stuff. Once the package is built, the resulting
artifact is then installed to the system in its appropriate place. :pep:`517`
defines the interface between the build backend & installer.

Broad overview of flow
======================
Expand Down Expand Up @@ -111,24 +111,24 @@ The package index gives pip a list of files for that package (via the existing P

pip chooses from the list a single file to download.

It may go back and choose another file to download
It may go back and choose another file to download.

When pip looks at the package index, the place where it looks has
basically a link. The link’s text is the name of the file
basically a link. The link’s text is the name of the file.

This is the `PyPI Simple API`_ (PyPI has several APIs, some are being
deprecated). pip looks at Simple API, documented initially at :pep:`503` --
packaging.python.org has PyPA specifications with more details for
Simple Repository API
Simple Repository API.

For this package name -- this is the list of files available
For this package name -- this is the list of files available.

Looks there for:

* The list of filenames
* Other info

Once it has those, selects one file, downloads it
Once it has those, it selects one file and downloads it.

(Question: If I want to ``pip install flask``, I think the whole list of filenames
cannot….should not be …. ? I want only the Flask …. Why am I getting the
Expand Down
2 changes: 1 addition & 1 deletion docs/html/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ distro community, cloud provider support channels, etc).

## Upgrading `pip`

Upgrading your `pip` by running:
Upgrade your `pip` by running:

```{pip-cli}
$ pip install --upgrade pip
Expand Down
2 changes: 1 addition & 1 deletion docs/html/topics/caching.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ You can use `pip cache dir` to get the cache directory that pip is currently con

### Default paths

````{tab} Unix
````{tab} Linux
```
~/.cache/pip
```
Expand Down
2 changes: 1 addition & 1 deletion docs/html/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -800,7 +800,7 @@ As noted previously, pip is a command line program. While it is implemented in
Python, and so is available from your Python code via ``import pip``, you must
not use pip's internal APIs in this way. There are a number of reasons for this:

#. The pip code assumes that is in sole control of the global state of the
#. The pip code assumes that it is in sole control of the global state of the
program.
pip manages things like the logging system configuration, or the values of
the standard IO streams, without considering the possibility that user code
Expand Down
3 changes: 3 additions & 0 deletions news/11381.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Implement logic to read the ``EXTERNALLY-MANAGED`` file as specified in PEP 668.
This allows a downstream Python distributor to prevent users from using pip to
modify the externally managed environment.
2 changes: 2 additions & 0 deletions news/11704.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix an issue when an already existing in-memory distribution would cause
exceptions in ``pip install``
15 changes: 15 additions & 0 deletions src/pip/_internal/commands/install.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
from pip._internal.utils.filesystem import test_writable_dir
from pip._internal.utils.logging import getLogger
from pip._internal.utils.misc import (
check_externally_managed,
ensure_dir,
get_pip_version,
protect_pip_from_modification_on_windows,
Expand Down Expand Up @@ -285,6 +286,20 @@ def run(self, options: Values, args: List[str]) -> int:
if options.use_user_site and options.target_dir is not None:
raise CommandError("Can not combine '--user' and '--target'")

# Check whether the environment we're installing into is externally
# managed, as specified in PEP 668. Specifying --root, --target, or
# --prefix disables the check, since there's no reliable way to locate
# the EXTERNALLY-MANAGED file for those cases. An exception is also
# made specifically for "--dry-run --report" for convenience.
installing_into_current_environment = (
not (options.dry_run and options.json_report_file)
and options.root_path is None
and options.target_dir is None
and options.prefix_path is None
)
if installing_into_current_environment:
check_externally_managed()

upgrade_strategy = "to-satisfy-only"
if options.upgrade:
upgrade_strategy = options.upgrade_strategy
Expand Down
7 changes: 6 additions & 1 deletion src/pip/_internal/commands/uninstall.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,10 @@
install_req_from_line,
install_req_from_parsed_requirement,
)
from pip._internal.utils.misc import protect_pip_from_modification_on_windows
from pip._internal.utils.misc import (
check_externally_managed,
protect_pip_from_modification_on_windows,
)

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -90,6 +93,8 @@ def run(self, options: Values, args: List[str]) -> int:
f'"pip help {self.name}")'
)

check_externally_managed()

protect_pip_from_modification_on_windows(
modifying_pip="pip" in reqs_to_uninstall
)
Expand Down
87 changes: 86 additions & 1 deletion src/pip/_internal/exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,14 @@
"""

import configparser
import contextlib
import locale
import logging
import pathlib
import re
import sys
from itertools import chain, groupby, repeat
from typing import TYPE_CHECKING, Dict, List, Optional, Union
from typing import TYPE_CHECKING, Dict, Iterator, List, Optional, Union

from pip._vendor.requests.models import Request, Response
from pip._vendor.rich.console import Console, ConsoleOptions, RenderResult
Expand All @@ -22,6 +27,8 @@
from pip._internal.metadata import BaseDistribution
from pip._internal.req.req_install import InstallRequirement

logger = logging.getLogger(__name__)


#
# Scaffolding
Expand Down Expand Up @@ -658,3 +665,81 @@ def __str__(self) -> str:
assert self.error is not None
message_part = f".\n{self.error}\n"
return f"Configuration file {self.reason}{message_part}"


_DEFAULT_EXTERNALLY_MANAGED_ERROR = f"""\
The Python environment under {sys.prefix} is managed externally, and may not be
manipulated by the user. Please use specific tooling from the distributor of
the Python installation to interact with this environment instead.
"""


class ExternallyManagedEnvironment(DiagnosticPipError):
"""The current environment is externally managed.
This is raised when the current environment is externally managed, as
defined by `PEP 668`_. The ``EXTERNALLY-MANAGED`` configuration is checked
and displayed when the error is bubbled up to the user.
:param error: The error message read from ``EXTERNALLY-MANAGED``.
"""

reference = "externally-managed-environment"

def __init__(self, error: Optional[str]) -> None:
if error is None:
context = Text(_DEFAULT_EXTERNALLY_MANAGED_ERROR)
else:
context = Text(error)
super().__init__(
message="This environment is externally managed",
context=context,
note_stmt=(
"If you believe this is a mistake, please contact your "
"Python installation or OS distribution provider."
),
hint_stmt=Text("See PEP 668 for the detailed specification."),
)

@staticmethod
def _iter_externally_managed_error_keys() -> Iterator[str]:
# LC_MESSAGES is in POSIX, but not the C standard. The most common
# platform that does not implement this category is Windows, where
# using other categories for console message localization is equally
# unreliable, so we fall back to the locale-less vendor message. This
# can always be re-evaluated when a vendor proposes a new alternative.
try:
category = locale.LC_MESSAGES
except AttributeError:
lang: Optional[str] = None
else:
lang, _ = locale.getlocale(category)
if lang is not None:
yield f"Error-{lang}"
for sep in ("-", "_"):
before, found, _ = lang.partition(sep)
if not found:
continue
yield f"Error-{before}"
yield "Error"

@classmethod
def from_config(
cls,
config: Union[pathlib.Path, str],
) -> "ExternallyManagedEnvironment":
parser = configparser.ConfigParser(interpolation=None)
try:
parser.read(config, encoding="utf-8")
section = parser["externally-managed"]
for key in cls._iter_externally_managed_error_keys():
with contextlib.suppress(KeyError):
return cls(section[key])
except KeyError:
pass
except (OSError, UnicodeDecodeError, configparser.ParsingError):
from pip._internal.utils._log import VERBOSE

exc_info = logger.isEnabledFor(VERBOSE)
logger.warning("Failed to read %s", config, exc_info=exc_info)
return cls(None)
2 changes: 1 addition & 1 deletion src/pip/_internal/locations/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
USER_CACHE_DIR = appdirs.user_cache_dir("pip")

# FIXME doesn't account for venv linked to global site-packages
site_packages: typing.Optional[str] = sysconfig.get_path("purelib")
site_packages: str = sysconfig.get_path("purelib")


def get_major_minor_version() -> str:
Expand Down
2 changes: 1 addition & 1 deletion src/pip/_internal/operations/check.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ def check_package_set(
if name not in package_set:
missed = True
if req.marker is not None:
missed = req.marker.evaluate()
missed = req.marker.evaluate({"extra": ""})
if missed:
missing_deps.add((name, req))
continue
Expand Down
6 changes: 5 additions & 1 deletion src/pip/_internal/req/req_install.py
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,11 @@ def __str__(self) -> str:
else:
s = "<InstallRequirement>"
if self.satisfied_by is not None:
s += " in {}".format(display_path(self.satisfied_by.location))
if self.satisfied_by.location is not None:
location = display_path(self.satisfied_by.location)
else:
location = "<memory>"
s += f" in {location}"
if self.comes_from:
if isinstance(self.comes_from, str):
comes_from: Optional[str] = self.comes_from
Expand Down
7 changes: 2 additions & 5 deletions src/pip/_internal/utils/egg_link.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
# The following comment should be removed at some point in the future.
# mypy: strict-optional=False

import os
import re
import sys
from typing import Optional
from typing import List, Optional

from pip._internal.locations import site_packages, user_site
from pip._internal.utils.virtualenv import (
Expand Down Expand Up @@ -57,7 +54,7 @@ def egg_link_path_from_location(raw_name: str) -> Optional[str]:
This method will just return the first one found.
"""
sites = []
sites: List[str] = []
if running_under_virtualenv():
sites.append(site_packages)
if not virtualenv_no_global() and user_site:
Expand Down
20 changes: 18 additions & 2 deletions src/pip/_internal/utils/misc.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import shutil
import stat
import sys
import sysconfig
import urllib.parse
from io import StringIO
from itertools import filterfalse, tee, zip_longest
Expand All @@ -38,7 +39,7 @@
from pip._vendor.tenacity import retry, stop_after_delay, wait_fixed

from pip import __version__
from pip._internal.exceptions import CommandError
from pip._internal.exceptions import CommandError, ExternallyManagedEnvironment
from pip._internal.locations import get_major_minor_version
from pip._internal.utils.compat import WINDOWS
from pip._internal.utils.virtualenv import running_under_virtualenv
Expand All @@ -57,10 +58,10 @@
"captured_stdout",
"ensure_dir",
"remove_auth_from_url",
"check_externally_managed",
"ConfiguredBuildBackendHookCaller",
]


logger = logging.getLogger(__name__)

T = TypeVar("T")
Expand Down Expand Up @@ -581,6 +582,21 @@ def protect_pip_from_modification_on_windows(modifying_pip: bool) -> None:
)


def check_externally_managed() -> None:
"""Check whether the current environment is externally managed.
If the ``EXTERNALLY-MANAGED`` config file is found, the current environment
is considered externally managed, and an ExternallyManagedEnvironment is
raised.
"""
if running_under_virtualenv():
return
marker = os.path.join(sysconfig.get_path("stdlib"), "EXTERNALLY-MANAGED")
if not os.path.isfile(marker):
return
raise ExternallyManagedEnvironment.from_config(marker)


def is_console_interactive() -> bool:
"""Is this console interactive?"""
return sys.stdin is not None and sys.stdin.isatty()
Expand Down
42 changes: 42 additions & 0 deletions tests/functional/test_install.py
Original file line number Diff line number Diff line change
Expand Up @@ -2351,3 +2351,45 @@ def test_install_8559_wheel_package_present(
allow_stderr_warning=False,
)
assert DEPRECATION_MSG_PREFIX not in result.stderr


@pytest.mark.skipif(
sys.version_info < (3, 11),
reason="3.11 required to find distributions via importlib metadata",
)
def test_install_existing_memory_distribution(script: PipTestEnvironment) -> None:
sitecustomize_text = textwrap.dedent(
"""
import sys
from importlib.metadata import Distribution, DistributionFinder
EXAMPLE_METADATA = '''Metadata-Version: 2.1
Name: example
Version: 1.0.0
'''
class ExampleDistribution(Distribution):
def locate_file(self, path):
return path
def read_text(self, filename):
if filename == 'METADATA':
return EXAMPLE_METADATA
class CustomFinder(DistributionFinder):
def find_distributions(self, context=None):
return [ExampleDistribution()]
sys.meta_path.append(CustomFinder())
"""
)
with open(script.site_packages_path / "sitecustomize.py", "w") as sitecustomize:
sitecustomize.write(sitecustomize_text)

result = script.pip("install", "example")

assert "Requirement already satisfied: example in <memory>" in result.stdout
Loading

0 comments on commit 8278b19

Please sign in to comment.