[24.1] The test suite no longer works offline

[mgorny]

Description

Up to 24.0, it was possible to run most of the test suite without Internet access, with only a handful of tests failing. However, in 24.1 a large number of tests (776, to be precise) are throwing errors when run in an environment without Internet access.

FWICS the problem stems from pip_editable_parts fixture that attempts to fetch and install setuptools:

__________________________________________ ERROR at setup of test_get_remote_url__no_remote ___________________________________________
[gw3] linux -- Python 3.11.9 /tmp/portage/dev-python/pip-24.1/work/pip-24.1-python3_11/install/usr/bin/python3.11

pip_src = PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip_src0/pip_src')
tmpdir_factory = TempPathFactory(_given_basetemp=PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3'...p/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3'), _retention_count=0, _retention_policy='all')

    @pytest.fixture(scope="session")
    def pip_editable_parts(
        pip_src: Path, tmpdir_factory: pytest.TempPathFactory
    ) -> Tuple[Path, ...]:
        pip_editable = tmpdir_factory.mktemp("pip") / "pip"
        shutil.copytree(pip_src, pip_editable, symlinks=True)
        # noxfile.py is Python 3 only
        assert compileall.compile_dir(
            pip_editable,
            quiet=1,
            rx=re.compile("noxfile.py$"),
        )
        pip_self_install_path = tmpdir_factory.mktemp("pip_self_install")
>       subprocess.check_call(
            [
                sys.executable,
                "-m",
                "pip",
                "install",
                "--target",
                pip_self_install_path,
                "-e",
                pip_editable,
            ]
        )

pip_editable = PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip0/pip')
pip_self_install_path = PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip_self_install0')
pip_src    = PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip_src0/pip_src')
tmpdir_factory = TempPathFactory(_given_basetemp=PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3'),
                _trace=<pluggy._tracing.TagTracerSub object at 0x7fd77d708dd0>,
                _basetemp=PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3'),
                _retention_count=0,
                _retention_policy='all')

tests/conftest.py:396: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

popenargs = (['/tmp/portage/dev-python/pip-24.1/work/pip-24.1-python3_11/install/usr/bin/python3.11', '-m', 'pip', 'install', '--target', PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip_self_install0'), ...],)
kwargs = {}, retcode = 1
cmd = ['/tmp/portage/dev-python/pip-24.1/work/pip-24.1-python3_11/install/usr/bin/python3.11', '-m', 'pip', 'install', '--target', PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip_self_install0'), ...]

    def check_call(*popenargs, **kwargs):
        """Run command with arguments.  Wait for command to complete.  If
        the exit code was zero then return, otherwise raise
        CalledProcessError.  The CalledProcessError object will have the
        return code in the returncode attribute.
    
        The arguments are the same as for the call function.  Example:
    
        check_call(["ls", "-l"])
        """
        retcode = call(*popenargs, **kwargs)
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
>           raise CalledProcessError(retcode, cmd)
E           subprocess.CalledProcessError: Command '['/tmp/portage/dev-python/pip-24.1/work/pip-24.1-python3_11/install/usr/bin/python3.11', '-m', 'pip', 'install', '--target', PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip_self_install0'), '-e', PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip0/pip')]' returned non-zero exit status 1.

cmd        = ['/tmp/portage/dev-python/pip-24.1/work/pip-24.1-python3_11/install/usr/bin/python3.11',
 '-m',
 'pip',
 'install',
 '--target',
 PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip_self_install0'),
 '-e',
 PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip0/pip')]
kwargs     = {}
popenargs  = (['/tmp/portage/dev-python/pip-24.1/work/pip-24.1-python3_11/install/usr/bin/python3.11',
  '-m',
  'pip',
  'install',
  '--target',
  PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip_self_install0'),
  '-e',
  PosixPath('/tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip0/pip')],)
retcode    = 1

/usr/lib/python3.11/subprocess.py:413: CalledProcessError
-------------------------------------------------------- Captured stdout setup --------------------------------------------------------
Obtaining file:///tmp/portage/dev-python/pip-24.1/temp/pytest-of-portage/pytest-0/popen-gw3/pip0/pip
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
-------------------------------------------------------- Captured stderr setup --------------------------------------------------------
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f5e8ef07550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/setuptools/
      WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f5e8ef0c110>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/setuptools/
      WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f5e8ef0cc50>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/setuptools/
      WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f5e8ef0d690>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/setuptools/
      WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f5e8ef0e350>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/setuptools/
      ERROR: Could not find a version that satisfies the requirement setuptools>=67.6.1 (from versions: none)
      ERROR: No matching distribution found for setuptools>=67.6.1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Expected behavior

(Most of the) tests working offline again.

pip version

24.1

Python version

3.11.9, 3.12.4

OS

Gentoo Linux amd64

How to Reproduce

1. Disable Internet access (e.g. using unshare -n).
2. Run pytest

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[mgorny]

FWICS, this could be resolved by adding --no-build-isolation, i.e.:

diff --git a/tests/conftest.py b/tests/conftest.py
index 35101cef2..5934e9f95 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -395,6 +395,7 @@ def pip_editable_parts(
             "-m",
             "pip",
             "install",
+            "--no-build-isolation",
             "--target",
             pip_self_install_path,
             "-e",

Does this look like an acceptable solution?

[ichard26]

As pip's build dependencies (setuptools and wheel) are included in tests/requirements.txt, adding --no-build-isolation seems fine. Feel free to file a PR.