gh-76785: Fixes for test.support.interpreters #112982

ericsnowcurrently · 2023-12-11T23:13:45Z

This involves a number of changes for PEP 734.

Issue: [subinterpreters] PEP 554 implementation: add interpreters module #76785

bedevere-bot · 2023-12-12T15:44:45Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Clang Installed 3.x has failed when building commit 86a77f4.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/531/builds/4959) and take a look at the build logs.
Check if the failure is related to this commit (86a77f4) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/531/builds/4959

Failed tests:

test_threading

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.13/test/libregrtest/single.py", line 178, in _runtest_env_changed_exc
    _load_run_test(result, runtests)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.13/test/libregrtest/single.py", line 125, in _load_run_test
    test_mod = importlib.import_module(module_name)
               ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1011, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.13/test/test_threading.py", line 30, in <module>
    from test.support import interpreters
ImportError: cannot import name 'interpreters' from 'test.support' (/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.clang-installed/build/target/lib/python3.13/test/support/__init__.py)

ericsnowcurrently · 2023-12-12T16:00:58Z

I have a fix for the buildbot failure in gh-113014.

bedevere-bot · 2023-12-12T16:03:27Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable Clang Installed 3.x has failed when building commit 86a77f4.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/350/builds/5021) and take a look at the build logs.
Check if the failure is related to this commit (86a77f4) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/350/builds/5021

Failed tests:

test_threading

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.13/test/libregrtest/single.py", line 178, in _runtest_env_changed_exc
    _load_run_test(result, runtests)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.13/test/libregrtest/single.py", line 125, in _load_run_test
    test_mod = importlib.import_module(module_name)
               ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1011, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.13/test/test_threading.py", line 30, in <module>
    from test.support import interpreters
ImportError: cannot import name 'interpreters' from 'test.support' (/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.13/test/support/__init__.py)

bedevere-bot · 2023-12-12T16:10:07Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable Clang Installed 3.x has failed when building commit 86a77f4.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/14/builds/5056) and take a look at the build logs.
Check if the failure is related to this commit (86a77f4) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/14/builds/5056

Failed tests:

test_threading

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.13/test/libregrtest/single.py", line 178, in _runtest_env_changed_exc
    _load_run_test(result, runtests)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.13/test/libregrtest/single.py", line 125, in _load_run_test
    test_mod = importlib.import_module(module_name)
               ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1011, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.13/test/test_threading.py", line 30, in <module>
    from test.support import interpreters
ImportError: cannot import name 'interpreters' from 'test.support' (/home/buildbot/buildarea/3.x.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.13/test/support/__init__.py)

gh-112982 broke test_threading on one of the s390 buildbots (Fedora Clang Installed). Apparently ImportError is raised (rather than ModuleNotFoundError) for the name part of "from" imports. This fixes that by catching ImportError in test_threading.py.

bedevere-bot · 2023-12-12T17:03:18Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable Clang Installed 3.x has failed when building commit 86a77f4.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/451/builds/3930) and take a look at the build logs.
Check if the failure is related to this commit (86a77f4) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/451/builds/3930

Failed tests:

test_threading

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.13/test/libregrtest/single.py", line 178, in _runtest_env_changed_exc
    _load_run_test(result, runtests)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.13/test/libregrtest/single.py", line 125, in _load_run_test
    test_mod = importlib.import_module(module_name)
               ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.13/importlib/__init__.py", line 88, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 1011, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.13/test/test_threading.py", line 30, in <module>
    from test.support import interpreters
ImportError: cannot import name 'interpreters' from 'test.support' (/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.13/test/support/__init__.py)

ezio-melotti · 2023-12-12T20:43:13Z

.github/CODEOWNERS

+Doc/**/*queue*                @rhettinger
+PCbuild/**/*queue*            @rhettinger
+Modules/_queuemodule.c        @rhettinger
+Lib/*queue*.py                @rhettinger
+Lib/asyncio/*queue*.py        @rhettinger
+Lib/multiprocessing/*queue*.py  @rhettinger
+Lib/test/*queue*.py           @rhettinger
+Lib/test_asyncio/*queue*.py   @rhettinger
+Lib/test_multiprocessing/*queue*.py  @rhettinger


The last matching pattern in CODEOWNERS takes the precedence, so you can do:

**/*queue* @rhettinger support/interpreters/queues.py @ericsnowcurrently

to set @rhettinger as owner of all the *queue* files except support/interpreters/queues.py, which will be owned by @ericsnowcurrently instead.

Moving the subinterpreter section after the **/*queue* line should also override the owner for support/interpreters/queues.py without further changes.

I thought there was something like that but I couldn't find it via web search. Thanks for explaining. I'll fix it in a follow-up PR.

@ezio-melotti

In gh-112982 I made some changes to .github/CODEOWNERS. Later, @ezio-melotti pointed out that some of those changes were unnecessary.

…GH-112982) This is so that we can run python -m test.test_interpreters. As such it backports that aspect of commit 86a77f4, where it is implemnented by a package __main__.py.

…GH-112982) This is so that we can run python -m test.test_interpreters. As such it backports that aspect of commit 86a77f4, where it is implemented by a package __main__.py.

bedevere-app · 2023-12-25T16:48:56Z

GH-113470 is a backport of this pull request to the 3.12 branch.

…H-113470) This is so that we can run python -m test.test_interpreters. As such it backports that aspect of commit 86a77f4, where it is implemented by a package __main__.py.

…GH-112982) (pythonGH-113470) This is so that we can run python -m test.test_interpreters. As such it backports that aspect of commit 86a77f4, where it is implemented by a package __main__.py. (cherry picked from commit bdad5c3) Co-authored-by: Jeff Allen <[email protected]>

…H-113470) (GH-113473) This is so that we can run python -m test.test_interpreters. As such it backports that aspect of commit 86a77f4, where it is implemented by a package __main__.py. (cherry picked from commit bdad5c3) Co-authored-by: Jeff Allen <[email protected]>

This involves a number of changes for PEP 734.

…ongh-113014) pythongh-112982 broke test_threading on one of the s390 buildbots (Fedora Clang Installed). Apparently ImportError is raised (rather than ModuleNotFoundError) for the name part of "from" imports. This fixes that by catching ImportError in test_threading.py.

@ezio-melotti

In pythongh-112982 I made some changes to .github/CODEOWNERS. Later, @ezio-melotti pointed out that some of those changes were unnecessary.

This involves a number of changes for PEP 734.

…ongh-113014) pythongh-112982 broke test_threading on one of the s390 buildbots (Fedora Clang Installed). Apparently ImportError is raised (rather than ModuleNotFoundError) for the name part of "from" imports. This fixes that by catching ImportError in test_threading.py.

@ezio-melotti

In pythongh-112982 I made some changes to .github/CODEOWNERS. Later, @ezio-melotti pointed out that some of those changes were unnecessary.

ericsnowcurrently added 17 commits November 28, 2023 14:45

Make Interpreter() idempotent.

76a7d7b

Interpreters are always isolated.

57a3c19

interp.id is always int.

747e542

Add InterpreterNotFoundError.

0ed9fb0

Stop using InterpreterID in _interpreters.

98f9cf3

Fix Interpreter.__repr__().

0277878

.run() -> .exec_sync()

59c8227

RunFailedError -> ExecFailure

fa132a2

Add Interpreter.run().

644efa5

Make the interpreters module a package.

abf2aa8

Add interpreters.Queue.

8ca303f

Add memoryview XID with _xxsubinterpreters import.

8b97373

Update CODEOWNERS.

cc20f1f

Ignore static builtin exception types.

cb0a605

Make CODEOWNERS more specific.

62a9bac

Fix submodule names.

6c71018

Use interpreters.__getattr__() for submodule aliases.

13a44e3

ericsnowcurrently added the skip news label Dec 11, 2023

ericsnowcurrently requested review from rhettinger, markshannon, gvanrossum, ezio-melotti and hugovk as code owners December 11, 2023 23:13

bedevere-app bot added the awaiting core review label Dec 11, 2023

bedevere-app bot mentioned this pull request Dec 11, 2023

[subinterpreters] PEP 554 implementation: add interpreters module #76785

Open

ericsnowcurrently removed request for rhettinger, markshannon and gvanrossum December 11, 2023 23:13

ericsnowcurrently added 2 commits December 11, 2023 16:18

Fix InterpreterIDTests.

c9d15d2

LONG_LONG_MAX -> LLONG_MAX

1c76c4a

Fix TestInterpreterRun.

778276f

ericsnowcurrently removed request for brettcannon, ncoghlan and warsaw December 12, 2023 15:24

ericsnowcurrently merged commit 86a77f4 into python:main Dec 12, 2023
30 checks passed

ericsnowcurrently deleted the pep-734-updates branch December 12, 2023 15:24

bedevere-app bot removed the awaiting core review label Dec 12, 2023

ericsnowcurrently mentioned this pull request Dec 12, 2023

gh-76785: Fix test_threading on Non-Subinterpreter Builds #113014

Merged

ezio-melotti reviewed Dec 12, 2023

View reviewed changes

ericsnowcurrently mentioned this pull request Dec 13, 2023

gh-76785: Fix CODEOWNERS #113038

Merged

ericsnowcurrently added a commit that referenced this pull request Dec 13, 2023

gh-76785: Fix CODEOWNERS (gh-113038)

480b4b3

In gh-112982 I made some changes to .github/CODEOWNERS. Later, @ezio-melotti pointed out that some of those changes were unnecessary.

jeff5 mentioned this pull request Dec 25, 2023

[3.12] gh-76785: Make test.test_interpreters executable (GH-112982) #113470

Merged

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024

pythongh-76785: Fixes for test.support.interpreters (pythongh-112982)

c00f567

This involves a number of changes for PEP 734.

aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024

pythongh-76785: Fix CODEOWNERS (pythongh-113038)

e40a54d

In pythongh-112982 I made some changes to .github/CODEOWNERS. Later, @ezio-melotti pointed out that some of those changes were unnecessary.

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024

pythongh-76785: Fixes for test.support.interpreters (pythongh-112982)

56d2ce0

This involves a number of changes for PEP 734.

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024

pythongh-76785: Fix CODEOWNERS (pythongh-113038)

8a0ff75

In pythongh-112982 I made some changes to .github/CODEOWNERS. Later, @ezio-melotti pointed out that some of those changes were unnecessary.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-76785: Fixes for test.support.interpreters #112982

gh-76785: Fixes for test.support.interpreters #112982

ericsnowcurrently commented Dec 11, 2023 •

edited by bedevere-app bot

Loading

bedevere-bot commented Dec 12, 2023

ericsnowcurrently commented Dec 12, 2023

bedevere-bot commented Dec 12, 2023

bedevere-bot commented Dec 12, 2023

bedevere-bot commented Dec 12, 2023

ezio-melotti Dec 12, 2023

ericsnowcurrently Dec 12, 2023

bedevere-app bot commented Dec 25, 2023

gh-76785: Fixes for test.support.interpreters #112982

gh-76785: Fixes for test.support.interpreters #112982

Conversation

ericsnowcurrently commented Dec 11, 2023 • edited by bedevere-app bot Loading

bedevere-bot commented Dec 12, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

ericsnowcurrently commented Dec 12, 2023

bedevere-bot commented Dec 12, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Dec 12, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

bedevere-bot commented Dec 12, 2023

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

ezio-melotti Dec 12, 2023

Choose a reason for hiding this comment

ericsnowcurrently Dec 12, 2023

Choose a reason for hiding this comment

bedevere-app bot commented Dec 25, 2023

ericsnowcurrently commented Dec 11, 2023 •

edited by bedevere-app bot

Loading