gh-87729: specialize LOAD_SUPER_ATTR_METHOD #103809

carljm · 2023-04-24T22:40:05Z

* main: pythongh-103479: [Enum] require __new__ to be considered a data type (pythonGH-103495) pythongh-103365: [Enum] STRICT boundary corrections (pythonGH-103494) pythonGH-103488: Use return-offset, not yield-offset. (pythonGH-103502) pythongh-103088: Fix test_venv error message to avoid bytes/str warning (pythonGH-103500) pythonGH-103082: Turn on branch events for FOR_ITER instructions. (python#103507) pythongh-102978: Fix mock.patch function signatures for class and staticmethod decorators (python#103228) pythongh-103462: Ensure SelectorSocketTransport.writelines registers a writer when data is still pending (python#103463) pythongh-95299: Rework test_cppext.py to not invoke setup.py directly (python#103316)

Co-authored-by: Dong-hee Na <[email protected]>

* main: pythongh-103532: Remove TKINTER_PROTECT_LOADTK code (pythonGH-103535) pythongh-103180: Add CI timeouts to all GitHub Actions jobs (python#103437) Remove double space in import error message (python#103458) ipaddress: Remove non-existent ip_str param from docstring (python#103461) Fix syntax typo in isolating extensions doc (python#103516) pythongh-103406: Modernize pos-only arguments usage in `test_signature` (python#103407) Proofread howto/perf_profiling.rst (python#103530) Fix unused functions warnings in instrumentation.c (pythonGH-103515)

* superopt: fix incompatible types update generated cases don't unnecessarily re-find args in error case Apply suggestions from code review pythongh-103532: Remove TKINTER_PROTECT_LOADTK code (pythonGH-103535) pythongh-103180: Add CI timeouts to all GitHub Actions jobs (python#103437) Remove double space in import error message (python#103458) ipaddress: Remove non-existent ip_str param from docstring (python#103461) Fix syntax typo in isolating extensions doc (python#103516) pythongh-103406: Modernize pos-only arguments usage in `test_signature` (python#103407) Proofread howto/perf_profiling.rst (python#103530) Fix unused functions warnings in instrumentation.c (pythonGH-103515)

* main: Remove `expert-*` from `project-updater` GH workflow (python#103579) pythongh-103583: Add codecs and maps to _codecs_* module state (python#103540) pythongh-48330: address review comments to PR-12271 (python#103209) pythongh-103527: Add multibytecodec.h as make dep for _codecs_* (python#103567) pythongh-103553: Improve `test_inspect`: add more assertions, remove unused (python#103554) pythonGH-103517: Improve tests for `pathlib.Path.walk()` (pythonGH-103518) pythongh-102114: Make dis print more concise tracebacks for syntax errors in str inputs (python#102115) pythonGH-78079: Fix UNC device path root normalization in pathlib (pythonGH-102003) pythongh-101517: Add regression test for a lineno bug in try/except* impacting pdb (python#103547) pythongh-103527: Add make deps for _codecs_* and _multibytecodec (python#103528) pythongh-103532: Fix reST syntax in NEWS entry (pythonGH-103544) pythongh-103532: Add NEWS entry (python#103542)

* superopt: update generated cases with new comment review comments Remove `expert-*` from `project-updater` GH workflow (python#103579) pythongh-103583: Add codecs and maps to _codecs_* module state (python#103540) pythongh-48330: address review comments to PR-12271 (python#103209) pythongh-103527: Add multibytecodec.h as make dep for _codecs_* (python#103567) pythongh-103553: Improve `test_inspect`: add more assertions, remove unused (python#103554) pythonGH-103517: Improve tests for `pathlib.Path.walk()` (pythonGH-103518) pythongh-102114: Make dis print more concise tracebacks for syntax errors in str inputs (python#102115) pythonGH-78079: Fix UNC device path root normalization in pathlib (pythonGH-102003) pythongh-101517: Add regression test for a lineno bug in try/except* impacting pdb (python#103547) pythongh-103527: Add make deps for _codecs_* and _multibytecodec (python#103528) pythongh-103532: Fix reST syntax in NEWS entry (pythonGH-103544) pythongh-103532: Add NEWS entry (python#103542)

* main: (24 commits) pythongh-98040: Move the Single-Phase Init Tests Out of test_imp (pythongh-102561) pythongh-83861: Fix datetime.astimezone() method (pythonGH-101545) pythongh-102856: Clean some of the PEP 701 tokenizer implementation (python#103634) pythongh-102856: Skip test_mismatched_parens in WASI builds (python#103633) pythongh-102856: Initial implementation of PEP 701 (python#102855) pythongh-103583: Add ref. dependency between multibytecodec modules (python#103589) pythongh-83004: Harden msvcrt further (python#103420) pythonGH-88342: clarify that `asyncio.as_completed` accepts generators yielding tasks (python#103626) pythongh-102778: IDLE - make sys.last_exc available in Shell after traceback (python#103314) pythongh-103582: Remove last references to `argparse.REMAINDER` from docs (python#103586) pythongh-103583: Always pass multibyte codec structs as const (python#103588) pythongh-103617: Fix compiler warning in _iomodule.c (python#103618) pythongh-103596: [Enum] do not shadow mixed-in methods/attributes (pythonGH-103600) pythonGH-100530: Change the error message for non-class class patterns (pythonGH-103576) pythongh-95299: Remove lingering setuptools reference in installer scripts (pythonGH-103613) [Doc] Fix a typo in optparse.rst (python#103504) pythongh-101100: Fix broken reference `__format__` in `string.rst` (python#103531) pythongh-95299: Stop installing setuptools as a part of ensurepip and venv (python#101039) pythonGH-103484: Docs: add linkcheck allowed redirects entries for most cases (python#103569) pythongh-67230: update whatsnew note for csv changes (python#103598) ...

* main: (53 commits) pythongh-102498 Clean up unused variables and imports in the email module (python#102482) pythongh-99184: Bypass instance attribute access in `repr` of `weakref.ref` (python#99244) pythongh-99032: datetime docs: Encoding is no longer relevant (python#93365) pythongh-94300: Update datetime.strptime documentation (python#95318) pythongh-103776: Remove explicit uses of $(SHELL) from Makefile (pythonGH-103778) pythongh-87092: fix a few cases of incorrect error handling in compiler (python#103456) pythonGH-103727: Avoid advancing tokenizer too far in f-string mode (pythonGH-103775) Revert "Add tests for empty range equality (python#103751)" (python#103770) pythongh-94518: Port 23-argument `_posixsubprocess.fork_exec` to Argument Clinic (python#94519) pythonGH-65022: Fix description of copyreg.pickle function (python#102656) pythongh-103323: Get the "Current" Thread State from a Thread-Local Variable (pythongh-103324) pythongh-91687: modernize dataclass example typing (python#103773) pythongh-103746: Test `types.UnionType` and `Literal` types together (python#103747) pythongh-103765: Fix 'Warning: py:class reference target not found: ModuleSpec' (pythonGH-103769) pythongh-87452: Improve the Popen.returncode docs Removed unnecessary escaping of asterisks (python#103714) pythonGH-102973: Slim down Fedora packages in the dev container (python#103283) pythongh-103091: Add PyUnstable_Type_AssignVersionTag (python#103095) Add tests for empty range equality (python#103751) pythongh-103712: Increase the length of the type name in AttributeError messages (python#103713) ...

* main: pythongh-101517: fix line number propagation in code generated for except* (python#103550) pythongh-103780: Use patch instead of mock in asyncio unix events test (python#103782)

* superopt: (82 commits) pythongh-101517: fix line number propagation in code generated for except* (python#103550) pythongh-103780: Use patch instead of mock in asyncio unix events test (python#103782) pythongh-102498 Clean up unused variables and imports in the email module (python#102482) pythongh-99184: Bypass instance attribute access in `repr` of `weakref.ref` (python#99244) pythongh-99032: datetime docs: Encoding is no longer relevant (python#93365) pythongh-94300: Update datetime.strptime documentation (python#95318) pythongh-103776: Remove explicit uses of $(SHELL) from Makefile (pythonGH-103778) pythongh-87092: fix a few cases of incorrect error handling in compiler (python#103456) pythonGH-103727: Avoid advancing tokenizer too far in f-string mode (pythonGH-103775) Revert "Add tests for empty range equality (python#103751)" (python#103770) pythongh-94518: Port 23-argument `_posixsubprocess.fork_exec` to Argument Clinic (python#94519) pythonGH-65022: Fix description of copyreg.pickle function (python#102656) pythongh-103323: Get the "Current" Thread State from a Thread-Local Variable (pythongh-103324) pythongh-91687: modernize dataclass example typing (python#103773) pythongh-103746: Test `types.UnionType` and `Literal` types together (python#103747) pythongh-103765: Fix 'Warning: py:class reference target not found: ModuleSpec' (pythonGH-103769) pythongh-87452: Improve the Popen.returncode docs Removed unnecessary escaping of asterisks (python#103714) pythonGH-102973: Slim down Fedora packages in the dev container (python#103283) pythongh-103091: Add PyUnstable_Type_AssignVersionTag (python#103095) ...

* main: pythongh-100227: Only Use deepfreeze for the Main Interpreter (pythongh-103794) pythongh-103492: Clarify SyntaxWarning with literal comparison (python#103493) pythongh-101100: Fix Sphinx warnings in `argparse` module (python#103289)

* superopt: pythongh-100227: Only Use deepfreeze for the Main Interpreter (pythongh-103794) pythongh-103492: Clarify SyntaxWarning with literal comparison (python#103493) pythongh-101100: Fix Sphinx warnings in `argparse` module (python#103289)

* main: pythongh-103801: Tools/wasm linting and formatting (python#103796) pythongh-103673: Add missing ForkingUnixStreamServer and ForkingUnixDatagramServer socketservers (python#103674) pythongh-95795: Move types.next_version_tag to PyInterpreterState (pythongh-102343)

bedevere-bot · 2023-04-24T23:29:03Z

🤖 New build scheduled with the buildbot fleet by @carljm for commit 3d0e51a 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

markshannon

There is some potential for simplifying LOAD_SUPER_ATTR, now that there is a specialization

Python/bytecodes.c

markshannon

Looks good now.
Thanks.

JelleZijlstra · 2023-04-26T14:51:03Z

Python/specialize.c

+        goto fail;
+    }
+    PyTypeObject *tp = (PyTypeObject *)class;
+    PyObject *res = _PySuper_LookupDescr(tp, self, name);


Is this leaked? I don't see where we DECREF it.

Yeah I think it's leaked, _PySuper_LookupDescr incref's its result.

For future reference, fix by @JelleZijlstra in #103882

tacaswell · 2023-04-27T22:52:47Z

I think this is breaking c++ extensions as class is a reserved keyword in c++

diff --git a/Include/internal/pycore_code.h b/Include/internal/pycore_code.h
index 7d5d5e03de..f15328fd8e 100644
--- a/Include/internal/pycore_code.h
+++ b/Include/internal/pycore_code.h
@@ -226,7 +226,7 @@ extern int _PyLineTable_PreviousAddressRange(PyCodeAddressRange *range);

 /* Specialization functions */

-extern void _Py_Specialize_LoadSuperAttr(PyObject *global_super, PyObject *class, PyObject *self,
+extern void _Py_Specialize_LoadSuperAttr(PyObject *global_super, PyObject *klass, PyObject *self,
                                          _Py_CODEUNIT *instr, PyObject *name, int load_method);
 extern void _Py_Specialize_LoadAttr(PyObject *owner, _Py_CODEUNIT *instr,
                                     PyObject *name);

gets things compiling again, but I am not sure if this is the right fix (or you want to propagate the name change through the implementation as well).

JelleZijlstra · 2023-04-27T23:16:24Z

We could just remove the parameter names from the prototype.

tacaswell · 2023-04-28T00:05:54Z

Just this prototype and leave the rest?

carljm · 2023-04-28T18:40:16Z

Thanks for the report!

All the other specialization functions include the parameter names in the prototype; it seems preferable to stay consistent. I don't mind changing class to klass throughout header and implementation; I can put up a PR for this.

carljm · 2023-04-28T18:48:22Z

fix PR in #103979

carljm and others added 30 commits April 12, 2023 20:12

pythongh-87729: add instruction for faster zero-arg super()

0a0ebe2

add news entry

f14a3bf

document LOAD_ZERO_SUPER_ATTR

5625e73

fix gdb test_wrapper_call

0775efe

optimize 2-arg super also

f229b5b

fix incompatible assignment

626999d

fix bad first arg

92c943b

specialize LOAD_SUPER_ATTR for methods

09804cd

Apply suggestions from code review

775ed0f

Co-authored-by: Dong-hee Na <[email protected]>

don't unnecessarily re-find args in error case

2077f1a

update generated cases

4759ad9

fix incompatible types

94399c2

don't double execute descriptor on specialization fail

5627d66

review comments

82945b2

update generated cases with new comment

3a3cb74

simplify oparg & 2 handling

e4466a7

cleanup and clarification

f161268

move __class__ special case out of the fast path

df442c0

Merge branch 'main' into superopt

0de5bc6

* main: pythongh-101517: fix line number propagation in code generated for except* (python#103550) pythongh-103780: Use patch instead of mock in asyncio unix events test (python#103782)

re-add comment

6858a6f

Merge branch 'main' into superopt

dbe1665

* main: pythongh-100227: Only Use deepfreeze for the Main Interpreter (pythongh-103794) pythongh-103492: Clarify SyntaxWarning with literal comparison (python#103493) pythongh-101100: Fix Sphinx warnings in `argparse` module (python#103289)

carljm requested review from brandtbucher and removed request for brettcannon, warsaw, encukou and ncoghlan April 24, 2023 22:53

carljm added 2 commits April 24, 2023 17:08

credit Vlad

3d0e51a

carljm requested a review from Fidget-Spinner April 24, 2023 23:09

carljm added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Apr 24, 2023

bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Apr 24, 2023

ambv and others added 3 commits April 25, 2023 02:08

Merge branch 'main' into superopt_spec

c96d5c1

Merge branch 'main' into superopt_spec

ad4008e

Merge branch 'main' into superopt_spec

66d6271

markshannon reviewed Apr 25, 2023

View reviewed changes

Python/bytecodes.c Show resolved Hide resolved

simplify LOAD_SUPER_ATTR, let specialization optimize

fc6fcea

markshannon approved these changes Apr 25, 2023

View reviewed changes

carljm enabled auto-merge (squash) April 25, 2023 17:30

carljm merged commit ef25feb into python:main Apr 25, 2023

carljm deleted the superopt_spec branch April 25, 2023 18:00

JelleZijlstra reviewed Apr 26, 2023

View reviewed changes

itamaro pushed a commit to itamaro/cpython that referenced this pull request Apr 26, 2023

pythongh-87729: specialize LOAD_SUPER_ATTR_METHOD (python#103809)

bf717d4

tacaswell mentioned this pull request Apr 27, 2023

MAINT: Allow scipy to be compiled in cython3 scipy/scipy#18242

Merged

5 tasks

matusvalo mentioned this pull request Apr 28, 2023

[BUG] Compilation error in cpp mode with python 3.12 caused by class C++ keyword in cpython header cython/cython#5409

Closed

carljm mentioned this pull request Apr 28, 2023

'class' parameter name in headers breaks building C++ extensions #103978

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-87729: specialize LOAD_SUPER_ATTR_METHOD #103809

gh-87729: specialize LOAD_SUPER_ATTR_METHOD #103809

carljm commented Apr 24, 2023

bedevere-bot commented Apr 24, 2023

markshannon left a comment

markshannon left a comment

JelleZijlstra Apr 26, 2023

Fidget-Spinner Apr 26, 2023

carljm Apr 26, 2023

tacaswell commented Apr 27, 2023

JelleZijlstra commented Apr 27, 2023

tacaswell commented Apr 28, 2023

carljm commented Apr 28, 2023 •

edited

Loading

carljm commented Apr 28, 2023

gh-87729: specialize LOAD_SUPER_ATTR_METHOD #103809

gh-87729: specialize LOAD_SUPER_ATTR_METHOD #103809

Conversation

carljm commented Apr 24, 2023

bedevere-bot commented Apr 24, 2023

markshannon left a comment

Choose a reason for hiding this comment

markshannon left a comment

Choose a reason for hiding this comment

JelleZijlstra Apr 26, 2023

Choose a reason for hiding this comment

Fidget-Spinner Apr 26, 2023

Choose a reason for hiding this comment

carljm Apr 26, 2023

Choose a reason for hiding this comment

tacaswell commented Apr 27, 2023

JelleZijlstra commented Apr 27, 2023

tacaswell commented Apr 28, 2023

carljm commented Apr 28, 2023 • edited Loading

carljm commented Apr 28, 2023

carljm commented Apr 28, 2023 •

edited

Loading