Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-118093: Make CALL_ALLOC_AND_ENTER_INIT suitable for tier 2. #123140

Merged
merged 11 commits into from
Aug 20, 2024

Conversation

markshannon
Copy link
Member

@markshannon markshannon commented Aug 19, 2024

This PR:
1 Breaks CALL_ALLOC_AND_ENTER_INIT into micro-ops, so that it can be included in tier 2 traces
2 Relaxes the requirement exact arguments, so that more calls to Python classes are inlined.
3 Passes the previous frame to the various PyInterpreterFrame creation functions.

(3) above seems more logical than setting the previous field separately, and is necessary to break up CALL_ALLOC_AND_ENTER_INIT as it pushes two frames.

@markshannon markshannon changed the title Make CALL_ALLOC_AND_ENTER_INIT suitable for tier 2. GH-118093: Make CALL_ALLOC_AND_ENTER_INIT suitable for tier 2. Aug 19, 2024
@markshannon
Copy link
Member Author

Stats show a small increase in tier 2 uops executed and the elimination of CALL_ALLOC_AND_ENTER_INIT as a reason to abandon projecting traces.

@markshannon markshannon merged commit bb1d303 into python:main Aug 20, 2024
57 checks passed
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL8 Refleaks 3.x has failed when building commit bb1d303.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/259/builds/1225) and take a look at the build logs.
  4. Check if the failure is related to this commit (bb1d303) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/259/builds/1225

Failed tests:

  • test_dataclasses

Summary of the results of the build (if available):

==

Click to see traceback logs
remote: Enumerating objects: 30, done.        
remote: Counting objects:   3% (1/30)        
remote: Counting objects:   6% (2/30)        
remote: Counting objects:  10% (3/30)        
remote: Counting objects:  13% (4/30)        
remote: Counting objects:  16% (5/30)        
remote: Counting objects:  20% (6/30)        
remote: Counting objects:  23% (7/30)        
remote: Counting objects:  26% (8/30)        
remote: Counting objects:  30% (9/30)        
remote: Counting objects:  33% (10/30)        
remote: Counting objects:  36% (11/30)        
remote: Counting objects:  40% (12/30)        
remote: Counting objects:  43% (13/30)        
remote: Counting objects:  46% (14/30)        
remote: Counting objects:  50% (15/30)        
remote: Counting objects:  53% (16/30)        
remote: Counting objects:  56% (17/30)        
remote: Counting objects:  60% (18/30)        
remote: Counting objects:  63% (19/30)        
remote: Counting objects:  66% (20/30)        
remote: Counting objects:  70% (21/30)        
remote: Counting objects:  73% (22/30)        
remote: Counting objects:  76% (23/30)        
remote: Counting objects:  80% (24/30)        
remote: Counting objects:  83% (25/30)        
remote: Counting objects:  86% (26/30)        
remote: Counting objects:  90% (27/30)        
remote: Counting objects:  93% (28/30)        
remote: Counting objects:  96% (29/30)        
remote: Counting objects: 100% (30/30)        
remote: Counting objects: 100% (30/30), done.        
remote: Compressing objects:   3% (1/30)        
remote: Compressing objects:   6% (2/30)        
remote: Compressing objects:  10% (3/30)        
remote: Compressing objects:  13% (4/30)        
remote: Compressing objects:  16% (5/30)        
remote: Compressing objects:  20% (6/30)        
remote: Compressing objects:  23% (7/30)        
remote: Compressing objects:  26% (8/30)        
remote: Compressing objects:  30% (9/30)        
remote: Compressing objects:  33% (10/30)        
remote: Compressing objects:  36% (11/30)        
remote: Compressing objects:  40% (12/30)        
remote: Compressing objects:  43% (13/30)        
remote: Compressing objects:  46% (14/30)        
remote: Compressing objects:  50% (15/30)        
remote: Compressing objects:  53% (16/30)        
remote: Compressing objects:  56% (17/30)        
remote: Compressing objects:  60% (18/30)        
remote: Compressing objects:  63% (19/30)        
remote: Compressing objects:  66% (20/30)        
remote: Compressing objects:  70% (21/30)        
remote: Compressing objects:  73% (22/30)        
remote: Compressing objects:  76% (23/30)        
remote: Compressing objects:  80% (24/30)        
remote: Compressing objects:  83% (25/30)        
remote: Compressing objects:  86% (26/30)        
remote: Compressing objects:  90% (27/30)        
remote: Compressing objects:  93% (28/30)        
remote: Compressing objects:  96% (29/30)        
remote: Compressing objects: 100% (30/30)        
remote: Compressing objects: 100% (30/30), done.        
remote: Total 30 (delta 4), reused 4 (delta 0), pack-reused 0 (from 0)        
From https://github.com/python/cpython
 * branch                    main       -> FETCH_HEAD
Note: switching to 'bb1d30336e83837d4191a016107fd501cd230328'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at bb1d30336e8 GH-118093: Make `CALL_ALLOC_AND_ENTER_INIT` suitable for tier 2. (GH-123140)
Switched to and reset branch 'main'

configure: WARNING: no system libmpdecimal found; falling back to bundled libmpdecimal (deprecated and scheduled for removal in Python 3.15)

make: *** [Makefile:2304: buildbottest] Error 2

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL9 Refleaks 3.x has failed when building commit bb1d303.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1589/builds/26) and take a look at the build logs.
  4. Check if the failure is related to this commit (bb1d303) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1589/builds/26

Failed tests:

  • test_dataclasses

Summary of the results of the build (if available):

==

Click to see traceback logs
remote: Enumerating objects: 30, done.        
remote: Counting objects:   3% (1/30)        
remote: Counting objects:   6% (2/30)        
remote: Counting objects:  10% (3/30)        
remote: Counting objects:  13% (4/30)        
remote: Counting objects:  16% (5/30)        
remote: Counting objects:  20% (6/30)        
remote: Counting objects:  23% (7/30)        
remote: Counting objects:  26% (8/30)        
remote: Counting objects:  30% (9/30)        
remote: Counting objects:  33% (10/30)        
remote: Counting objects:  36% (11/30)        
remote: Counting objects:  40% (12/30)        
remote: Counting objects:  43% (13/30)        
remote: Counting objects:  46% (14/30)        
remote: Counting objects:  50% (15/30)        
remote: Counting objects:  53% (16/30)        
remote: Counting objects:  56% (17/30)        
remote: Counting objects:  60% (18/30)        
remote: Counting objects:  63% (19/30)        
remote: Counting objects:  66% (20/30)        
remote: Counting objects:  70% (21/30)        
remote: Counting objects:  73% (22/30)        
remote: Counting objects:  76% (23/30)        
remote: Counting objects:  80% (24/30)        
remote: Counting objects:  83% (25/30)        
remote: Counting objects:  86% (26/30)        
remote: Counting objects:  90% (27/30)        
remote: Counting objects:  93% (28/30)        
remote: Counting objects:  96% (29/30)        
remote: Counting objects: 100% (30/30)        
remote: Counting objects: 100% (30/30), done.        
remote: Compressing objects:   3% (1/30)        
remote: Compressing objects:   6% (2/30)        
remote: Compressing objects:  10% (3/30)        
remote: Compressing objects:  13% (4/30)        
remote: Compressing objects:  16% (5/30)        
remote: Compressing objects:  20% (6/30)        
remote: Compressing objects:  23% (7/30)        
remote: Compressing objects:  26% (8/30)        
remote: Compressing objects:  30% (9/30)        
remote: Compressing objects:  33% (10/30)        
remote: Compressing objects:  36% (11/30)        
remote: Compressing objects:  40% (12/30)        
remote: Compressing objects:  43% (13/30)        
remote: Compressing objects:  46% (14/30)        
remote: Compressing objects:  50% (15/30)        
remote: Compressing objects:  53% (16/30)        
remote: Compressing objects:  56% (17/30)        
remote: Compressing objects:  60% (18/30)        
remote: Compressing objects:  63% (19/30)        
remote: Compressing objects:  66% (20/30)        
remote: Compressing objects:  70% (21/30)        
remote: Compressing objects:  73% (22/30)        
remote: Compressing objects:  76% (23/30)        
remote: Compressing objects:  80% (24/30)        
remote: Compressing objects:  83% (25/30)        
remote: Compressing objects:  86% (26/30)        
remote: Compressing objects:  90% (27/30)        
remote: Compressing objects:  93% (28/30)        
remote: Compressing objects:  96% (29/30)        
remote: Compressing objects: 100% (30/30)        
remote: Compressing objects: 100% (30/30), done.        
remote: Total 30 (delta 4), reused 4 (delta 0), pack-reused 0 (from 0)        
From https://github.com/python/cpython
 * branch                    main       -> FETCH_HEAD
Note: switching to 'bb1d30336e83837d4191a016107fd501cd230328'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at bb1d30336e8 GH-118093: Make `CALL_ALLOC_AND_ENTER_INIT` suitable for tier 2. (GH-123140)
Switched to and reset branch 'main'

configure: WARNING: no system libmpdecimal found; falling back to bundled libmpdecimal (deprecated and scheduled for removal in Python 3.15)

./Modules/blake2module.c: In function ‘type_to_impl’:
./Modules/blake2module.c:314:18: warning: unused variable ‘st’ [-Wunused-variable]
  314 |     Blake2State* st = blake2_get_state_from_type(type);
      |                  ^~

make: *** [Makefile:2300: buildbottest] Error 2

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Refleaks 3.x has failed when building commit bb1d303.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/433/builds/1225) and take a look at the build logs.
  4. Check if the failure is related to this commit (bb1d303) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/433/builds/1225

Failed tests:

  • test_dataclasses

Summary of the results of the build (if available):

==

Click to see traceback logs
remote: Enumerating objects: 30, done.        
remote: Counting objects:   3% (1/30)        
remote: Counting objects:   6% (2/30)        
remote: Counting objects:  10% (3/30)        
remote: Counting objects:  13% (4/30)        
remote: Counting objects:  16% (5/30)        
remote: Counting objects:  20% (6/30)        
remote: Counting objects:  23% (7/30)        
remote: Counting objects:  26% (8/30)        
remote: Counting objects:  30% (9/30)        
remote: Counting objects:  33% (10/30)        
remote: Counting objects:  36% (11/30)        
remote: Counting objects:  40% (12/30)        
remote: Counting objects:  43% (13/30)        
remote: Counting objects:  46% (14/30)        
remote: Counting objects:  50% (15/30)        
remote: Counting objects:  53% (16/30)        
remote: Counting objects:  56% (17/30)        
remote: Counting objects:  60% (18/30)        
remote: Counting objects:  63% (19/30)        
remote: Counting objects:  66% (20/30)        
remote: Counting objects:  70% (21/30)        
remote: Counting objects:  73% (22/30)        
remote: Counting objects:  76% (23/30)        
remote: Counting objects:  80% (24/30)        
remote: Counting objects:  83% (25/30)        
remote: Counting objects:  86% (26/30)        
remote: Counting objects:  90% (27/30)        
remote: Counting objects:  93% (28/30)        
remote: Counting objects:  96% (29/30)        
remote: Counting objects: 100% (30/30)        
remote: Counting objects: 100% (30/30), done.        
remote: Compressing objects:   3% (1/30)        
remote: Compressing objects:   6% (2/30)        
remote: Compressing objects:  10% (3/30)        
remote: Compressing objects:  13% (4/30)        
remote: Compressing objects:  16% (5/30)        
remote: Compressing objects:  20% (6/30)        
remote: Compressing objects:  23% (7/30)        
remote: Compressing objects:  26% (8/30)        
remote: Compressing objects:  30% (9/30)        
remote: Compressing objects:  33% (10/30)        
remote: Compressing objects:  36% (11/30)        
remote: Compressing objects:  40% (12/30)        
remote: Compressing objects:  43% (13/30)        
remote: Compressing objects:  46% (14/30)        
remote: Compressing objects:  50% (15/30)        
remote: Compressing objects:  53% (16/30)        
remote: Compressing objects:  56% (17/30)        
remote: Compressing objects:  60% (18/30)        
remote: Compressing objects:  63% (19/30)        
remote: Compressing objects:  66% (20/30)        
remote: Compressing objects:  70% (21/30)        
remote: Compressing objects:  73% (22/30)        
remote: Compressing objects:  76% (23/30)        
remote: Compressing objects:  80% (24/30)        
remote: Compressing objects:  83% (25/30)        
remote: Compressing objects:  86% (26/30)        
remote: Compressing objects:  90% (27/30)        
remote: Compressing objects:  93% (28/30)        
remote: Compressing objects:  96% (29/30)        
remote: Compressing objects: 100% (30/30)        
remote: Compressing objects: 100% (30/30), done.        
remote: Total 30 (delta 4), reused 4 (delta 0), pack-reused 0 (from 0)        
From https://github.com/python/cpython
 * branch                    main       -> FETCH_HEAD
Note: switching to 'bb1d30336e83837d4191a016107fd501cd230328'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at bb1d30336e8 GH-118093: Make `CALL_ALLOC_AND_ENTER_INIT` suitable for tier 2. (GH-123140)
Switched to and reset branch 'main'

configure: WARNING: no system libmpdecimal found; falling back to bundled libmpdecimal (deprecated and scheduled for removal in Python 3.15)

./Modules/blake2module.c: In function ‘type_to_impl’:
./Modules/blake2module.c:314:18: warning: unused variable ‘st’ [-Wunused-variable]
  314 |     Blake2State* st = blake2_get_state_from_type(type);
      |                  ^~

make: *** [Makefile:2302: buildbottest] Error 2

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 Refleaks 3.x has failed when building commit bb1d303.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/75/builds/1228) and take a look at the build logs.
  4. Check if the failure is related to this commit (bb1d303) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/75/builds/1228

Failed tests:

  • test_dataclasses

Failed subtests:

  • test_address - test.test_socket.CreateServerTest.test_address

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.refleak/build/Lib/test/test_socket.py", line 6861, in test_address
    with socket.create_server(("::1", port),
         ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
                              family=socket.AF_INET6) as sock:
                              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.refleak/build/Lib/socket.py", line 947, in create_server
    raise error(err.errno, msg) from None
OSError: [Errno 98] Address already in use (while attempting to bind on address ('::1', 54943))

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable Refleaks 3.x has failed when building commit bb1d303.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/320/builds/1180) and take a look at the build logs.
  4. Check if the failure is related to this commit (bb1d303) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/320/builds/1180

Failed tests:

  • test_dataclasses
  • test_glob

Failed subtests:

  • test_selflink - test.test_glob.SymlinkLoopGlobTests.test_selflink

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/test/test_glob.py", line 543, in test_selflink
    self.assertIn(path, results)
    ~~~~~~~~~~~~~^^^^^^^^^^^^^^^
AssertionError: 'dir/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/file' not found in {'dir/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/file', 'dir/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/file', 'dir/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/file', 'dir/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/file', 'dir/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/link/file'}

@encukou
Copy link
Member

encukou commented Aug 21, 2024

As @Eclips4 wrote in #123185, this made code like this crash:

class Foo:
    def __init__(self, arg): pass

for _ in range(2):
    try:
        Foo()
    except:
        pass

@markshannon markshannon deleted the call-class-tier-2 branch August 21, 2024 09:53
blhsing pushed a commit to blhsing/cpython that referenced this pull request Aug 22, 2024
pythonGH-123140)

* Convert CALL_ALLOC_AND_ENTER_INIT to micro-ops such that tier 2 supports it

* Allow inexact arguments for CALL_ALLOC_AND_ENTER_INIT.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants