Segmentation fault with compile

[shadchin]

Crash report

What happened?

Minimal reproducible example

tests.py

def mocks(self, search_rv=None, mock_internal_search=False):
    with mock.patch.object(
        TestResourceController, "_search" if mock_internal_search else "search", return_value=search_rv
    ) as mock_search, mock.patch.object(TestResourceController, "_count") as mock_count, mock.patch.object(
        TestResourceController, "_before_create"
    ) as mock_before_create, mock.patch.object(
        TestResourceController, "_create"
    ) as mock_create, mock.patch.object(
        TestResourceController, "_after_create"
    ) as mock_after_create, mock.patch.object(
        TestResourceController, "_before_update"
    ) as mock_before_update, mock.patch.object(
        TestResourceController, "_update"
    ) as mock_update, mock.patch.object(
        TestResourceController, "_after_update"
    ) as mock_after_update, mock.patch.object(
        TestResourceController, "_before_delete"
    ) as mock_before_delete, mock.patch.object(
        TestResourceController, "_delete"
    ) as mock_delete, mock.patch.object(
        TestResourceController, "_after_delete"
    ) as mock_after_delete, mock.patch.object(
        TestResourceController, "validate"
    ) as mock_validate, mock.patch.object(
        TestSourceManager, "search", side_effect=self.mock_source_search_func()
    ) as mock_source_search, mock.patch.object(
        TestSourceManager, "before_create"
    ) as mock_source_before_create, mock.patch.object(
        TestSourceManager, "after_create"
    ) as mock_source_after_create, mock.patch.object(
        TestSourceManager, "before_update"
    ) as mock_source_before_update, mock.patch.object(
        TestSourceManager, "after_update"
    ) as mock_source_after_update, mock.patch.object(
        TestSourceManager, "before_delete"
    ) as mock_source_before_delete, mock.patch.object(
        TestSourceManager, "after_delete"
    ) as mock_source_after_delete, mock.patch.object(
        TestSourceManager2, "search", side_effect=self.mock_source_search_func()
    ) as mock_source2_search:
        yield 1

crash.py

with open("tests.py") as f:
    source = f.read()

compile(source, "tests.py", "exec")

Segmentation fault

shadchin@i113735019 ~ % python3.12 crash.py
zsh: segmentation fault  python3.12 crash.py

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   Python                        	       0x1047dc148 _PyCfg_OptimizeCodeUnit + 348
1   Python                        	       0x1047c1eb0 optimize_and_assemble + 252
2   Python                        	       0x1047c9298 compiler_function + 584
3   Python                        	       0x1047c2a30 compiler_body + 208
4   Python                        	       0x1047c0c70 compiler_codegen + 148
5   Python                        	       0x1047c02d8 _PyAST_Compile + 44
6   Python                        	       0x10480c81c Py_CompileStringObject + 112
7   Python                        	       0x1047a881c builtin_compile + 508
8   Python                        	       0x104711e34 cfunction_vectorcall_FASTCALL_KEYWORDS + 92
9   Python                        	       0x1047b7820 _PyEval_EvalFrameDefault + 43368
10  Python                        	       0x1047acc48 PyEval_EvalCode + 184
11  Python                        	       0x10480e4d0 run_eval_code_obj + 88
12  Python                        	       0x10480c6a8 run_mod + 132
13  Python                        	       0x10480bbb0 pyrun_file + 148
14  Python                        	       0x10480af8c _PyRun_SimpleFileObject + 288
15  Python                        	       0x10480ac64 _PyRun_AnyFileObject + 232
16  Python                        	       0x10482fa14 pymain_run_file_obj + 220
17  Python                        	       0x10482f674 pymain_run_file + 72
18  Python                        	       0x10482ee0c Py_RunMain + 860
19  Python                        	       0x10482ef50 Py_BytesMain + 40
20  dyld                          	       0x195463f28 start + 2236


Thread 0 crashed with ARM Thread State (64-bit):
    x0: 0x00000001042e4500   x1: 0x00000000000000b0   x2: 0x0000000000000000   x3: 0x000000000000003f
    x4: 0x0000000000000020   x5: 0x000000000000003e   x6: 0x0000000000000000   x7: 0x000000000000001c
    x8: 0x000000000000000c   x9: 0x00000000042d26f0  x10: 0x0000000000000660  x11: 0x00000001048cf440
   x12: 0x0000000000000028  x13: 0x0000000000000001  x14: 0x0000000104aff8c0  x15: 0x0000000000000001
   x16: 0x0000000000000008  x17: 0x0000600000acc090  x18: 0x0000000000000000  x19: 0x0000000142648960
   x20: 0x00000001426380d0  x21: 0x000000016bdb2ad0  x22: 0x0000000000000000  x23: 0x0208805008000001
   x24: 0x00000001042d26a0  x25: 0x00000001426488b0  x26: 0x0000000104183470  x27: 0x0000000000000000
   x28: 0x00000001042d26f0   fp: 0x000000016bdb2ac0   lr: 0x00000001047dc314
    sp: 0x000000016bdb29a0   pc: 0x00000001047dc148 cpsr: 0x20001000
   far: 0x0000000125816bf0  esr: 0x92000006 (Data Abort) byte read Translation fault

CPython versions tested on:

3.12

Operating systems tested on:

Linux, macOS

Output from running 'python -VV' on the command line:

Python 3.12.1 (main, Dec 8 2023, 18:57:37) [Clang 14.0.3 (clang-1403.0.22.14.1)]

Linked PRs

• gh-113297: Fix segfault in compiler for with statement with 19 context managers #113327
• [3.12] gh-113297: Fix segfault in compiler for with statement with 19 context managers (#113327) #113404

[Eclips4]

Hello! Thanks for the report.
Confirmed on current main.

FWIW, it's reproducible without a compile:

Details

# example.py

def mocks(self, search_rv=None, mock_internal_search=False):
    with mock.patch.object(
        TestResourceController, "_search" if mock_internal_search else "search", return_value=search_rv
    ) as mock_search, mock.patch.object(TestResourceController, "_count") as mock_count, mock.patch.object(
        TestResourceController, "_before_create"
    ) as mock_before_create, mock.patch.object(
        TestResourceController, "_create"
    ) as mock_create, mock.patch.object(
        TestResourceController, "_after_create"
    ) as mock_after_create, mock.patch.object(
        TestResourceController, "_before_update"
    ) as mock_before_update, mock.patch.object(
        TestResourceController, "_update"
    ) as mock_update, mock.patch.object(
        TestResourceController, "_after_update"
    ) as mock_after_update, mock.patch.object(
        TestResourceController, "_before_delete"
    ) as mock_before_delete, mock.patch.object(
        TestResourceController, "_delete"
    ) as mock_delete, mock.patch.object(
        TestResourceController, "_after_delete"
    ) as mock_after_delete, mock.patch.object(
        TestResourceController, "validate"
    ) as mock_validate, mock.patch.object(
        TestSourceManager, "search", side_effect=self.mock_source_search_func()
    ) as mock_source_search, mock.patch.object(
        TestSourceManager, "before_create"
    ) as mock_source_before_create, mock.patch.object(
        TestSourceManager, "after_create"
    ) as mock_source_after_create, mock.patch.object(
        TestSourceManager, "before_update"
    ) as mock_source_before_update, mock.patch.object(
        TestSourceManager, "after_update"
    ) as mock_source_after_update, mock.patch.object(
        TestSourceManager, "before_delete"
    ) as mock_source_before_delete, mock.patch.object(
        TestSourceManager, "after_delete"
    ) as mock_source_after_delete, mock.patch.object(
        TestSourceManager2, "search", side_effect=self.mock_source_search_func()
    ) as mock_source2_search:
        yield 1

./python example.py
> Segmentation fault

[tonybaloney]

It's an out of range array error at the C level, stack->depth is 52152544 here https://github.com/python/cpython/blame/main/Python/flowgraph.c#L676

Tagging @iritkatriel

Stack trace:

python.exe!except_stack_top (/Users/anthonyshaw/projects/cpython/Python/flowgraph.c:676)
python.exe!label_exception_targets (/Users/anthonyshaw/projects/cpython/Python/flowgraph.c:843)
python.exe!_PyCfg_OptimizeCodeUnit (/Users/anthonyshaw/projects/cpython/Python/flowgraph.c:2416)
python.exe!optimize_and_assemble_code_unit (/Users/anthonyshaw/projects/cpython/Python/compile.c:7597)
python.exe!optimize_and_assemble (/Users/anthonyshaw/projects/cpython/Python/compile.c:7639)
python.exe!compiler_function_body (/Users/anthonyshaw/projects/cpython/Python/compile.c:2322)
python.exe!compiler_function (/Users/anthonyshaw/projects/cpython/Python/compile.c:2423)
python.exe!compiler_visit_stmt (Unknown Source:0)
python.exe!compiler_body (/Users/anthonyshaw/projects/cpython/Python/compile.c:1755)
python.exe!compiler_codegen (/Users/anthonyshaw/projects/cpython/Python/compile.c:1771)
python.exe!compiler_mod (/Users/anthonyshaw/projects/cpython/Python/compile.c:1799)
python.exe!_PyAST_Compile (/Users/anthonyshaw/projects/cpython/Python/compile.c:555)
python.exe!Py_CompileStringObject (/Users/anthonyshaw/projects/cpython/Python/pythonrun.c:1452)
python.exe!builtin_compile_impl (/Users/anthonyshaw/projects/cpython/Python/bltinmodule.c:855)
python.exe!builtin_compile (/Users/anthonyshaw/projects/cpython/Python/clinic/bltinmodule.c.h:380)
python.exe!cfunction_vectorcall_FASTCALL_KEYWORDS (/Users/anthonyshaw/projects/cpython/Objects/methodobject.c:441)
python.exe!_PyObject_VectorcallTstate (/Users/anthonyshaw/projects/cpython/Include/internal/pycore_call.h:168)
python.exe!PyObject_Vectorcall (/Users/anthonyshaw/projects/cpython/Objects/call.c:327)
python.exe!_PyEval_EvalFrameDefault (/Users/anthonyshaw/projects/cpython/Python/generated_cases.c.h:795)
python.exe!_PyEval_EvalFrame (/Users/anthonyshaw/projects/cpython/Include/internal/pycore_ceval.h:116)

[Eclips4]

Reduced MRE:

def bug():
    with (
    a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a, a
    as a
    ):
        yield a

[iritkatriel]

@Eclips4 Have you reproduced it on 3.12? I see it on main but not 3.12.

[iritkatriel]

Ah, I can reproduce on 3.12.1+, but not on 3.12.0a0.

[Eclips4]

I cannot judge how this solution is true, but changing this line

cpython/Python/flowgraph.c

Line 651 in b221e03

basicblock *handlers[CO_MAXBLOCKS+1];

to this: basicblock *handlers[CO_MAXBLOCKS+2]; solves the segfault.

[iritkatriel]

This is the same as @Eclips4 's repro (just numbered):

def bug():
    with (
    a
    as a1, a
    as a2, a
    as a3, a
    as a4, a
    as a5, a
    as a6, a
    as a7, a
    as a8, a
    as a9, a
    as a10, a
    as a11, a
    as a12, a
    as a13, a
    as a14, a
    as a15, a
    as a16, a
    as a17, a
    as a18, a
    as a19, a
    as a
    ):
        yield a

If I add one more context manager, it doesn't crash but raises a syntax error about exceeding static depth:

def bug():
    with (
    a
    as a1, a
    as a2, a
    as a3, a
    as a4, a
    as a5, a
    as a6, a
    as a7, a
    as a8, a
    as a9, a
    as a10, a
    as a11, a
    as a12, a
    as a13, a
    as a14, a
    as a15, a
    as a16, a
    as a17, a
    as a18, a
    as a19, a
    as a20, a
    as a
    ):
        yield a

% ./python.exe tt.py
  File "/Users/iritkatriel/src/cpython-1/tt.py", line 2
    with (
    ^^^^^^^
SyntaxError: too many statically nested blocks

The bug is that the compiler allocates space for one too few elements in the exception stack. I will make a PR to fix this.

[iritkatriel]

I cannot judge how this solution is true, but changing this line

cpython/Python/flowgraph.c

Line 651 in b221e03

basicblock *handlers[CO_MAXBLOCKS+1];

to this: basicblock *handlers[CO_MAXBLOCKS+2]; solves the segfault.

We crossed wires - yes that's the one.