Use sysconfig instead of distutils for Python 3.10 compatability

[9999years]

• distutils is deprecated in Python 3.10 and slated for removal in Python 3.12
• sysconfig provides the same functionality

This is causing a bunch of warnings when running tox under Python 3.10:

> poetry310 run tox -e py310 --recreate
/my-venv-dir/lib/python3.10/site-packages/tox/helper/get_site_package_dir.py:3: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  import distutils.sysconfig
/my-venv-dir/lib/python3.10/site-packages/tox/helper/get_site_package_dir.py:3: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
  import distutils.sysconfig
/my-venv-dir/lib/python3.10/site-packages/tox/helper/get_site_package_dir.py:3: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  import distutils.sysconfig
/my-venv-dir/lib/python3.10/site-packages/tox/helper/get_site_package_dir.py:3: DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
  import distutils.sysconfig
.package recreate: my-repo-path/.tox/.package

[gaborbernat]

• distutils is deprecated in Python 3.10 and slated for removal in Python 3.12

Indeed.

• sysconfig provides the same functionality

Sadly this is not true. Especially not true on Linux distributions where the OS patches often distutils, but not sysconfig. While we might need to do this the change itself will break people, so likely better done in a new major version 🤔

[9999years]

likely better done in a new major version

OK, let's get it in for 4.0 then. How do we move forward with that?

[9999years]

Especially not true on Linux distributions where the OS patches often distutils, but not sysconfig.

I'm curious about this. Here's a test:

import distutils.sysconfig
import sysconfig


def check_purelib_path(prefix: str):
    """Assert that two methods of getting the Python package installation path
    (``distutils.sysconfig.get_python_lib`` and ``sysconfig.get_path``) return
    the same result.
    """
    distutils_path = distutils.sysconfig.get_python_lib(prefix=prefix)
    sysconfig_path = sysconfig.get_path("purelib", vars={"base": prefix})
    assert distutils_path == sysconfig_path, f"{distutils_path} != {sysconfig_path}"

Do you know of any distributions where check_purelib_path("...") fails in general?

I discovered these edge cases:

distutils.sysconfig.get_python_lib(prefix="/")    == "/lib/python3.9/site-packages"
sysconfig.get_path("purelib", vars={"base": "/"}) == "//lib/python3.9/site-packages"

distutils.sysconfig.get_python_lib(prefix="")     == "lib/python3.9/site-packages"
sysconfig.get_path("purelib", vars={"base": ""})  == "/lib/python3.9/site-packages"

[gaborbernat]

Do you know of any distributions where check_purelib_path("...") fails in general?

Can't recall exactly now, but now we've run into this issue on some Debian derivatives.

[gaborbernat]

Let's see if this will cause any major issues 🤷

[asottile]

Do you know of any distributions where check_purelib_path("...") fails in general?

Can't recall exactly now, but now we've run into this issue on some Debian derivatives.

yeah I think this is going to break on installed-to-system debian-derivatives

specifically:

$ python3 t.py 
Traceback (most recent call last):
  File "t.py", line 14, in <module>
    check_purelib_path("/usr")
  File "t.py", line 12, in check_purelib_path
    assert distutils_path == sysconfig_path, f"{distutils_path} != {sysconfig_path}"
AssertionError: /usr/lib/python3/dist-packages != /usr/lib/python3.8/site-packages
$ cat t.py 
import distutils.sysconfig
import sysconfig


def check_purelib_path(prefix: str):
    """Assert that two methods of getting the Python package installation path
    (``distutils.sysconfig.get_python_lib`` and ``sysconfig.get_path``) return
    the same result.
    """
    distutils_path = distutils.sysconfig.get_python_lib(prefix=prefix)
    sysconfig_path = sysconfig.get_path("purelib", vars={"base": prefix})
    assert distutils_path == sysconfig_path, f"{distutils_path} != {sysconfig_path}"

check_purelib_path("/usr")

[9999years]

@asottile Is there any way to resolve this other than getting debian maintainers to fix their patches / do you have a workaround? I don't have a debian system to test on.

[asottile]

not that I know of

[gaborbernat]

Went with this more forgiving fix in the end 91dfd3e

[gaborbernat]

Released via https://pypi.org/project/tox/3.24.0/