gh-117139: Convert the evaluation stack to stack refs #118450
Conversation
Fidget-Spinner
requested review from
a team,
gvanrossum,
kumaraditya303,
brettcannon,
ericsnowcurrently,
ncoghlan,
warsaw,
markshannon,
methane and
brandtbucher
as code owners
|
Could we hold off on this until 3.14? It's only a week until feature freeze for 3.13 (at which point main becomes 3.14), and this looks like a lot of churn in a time where we all would like stability to merge things that are actually needed in 3.13. |
Alright. I always forget that ten other people are rushing in things at the same time as me. I don't think this will add any value for CPython 3.13 anyways at the moment. |
That makes sense. I'll start providing feedback and reviewing this now, but it won't be merged in 3.13. |
|
@markshannon the last legitimately failing buildbot now passes https://buildbot.python.org/all/#/builders/1364/builds/150 ! It seems ASAN actually caught a real bug there, where vectorcall implicitly assumes the -1 entry of a stack is available for writing |
|
I'm going to merge this in an hour. We can keep iterating on it if needed after. |
Fidget-Spinner
dismissed
markshannon’s stale review
All addressed, or can be done in follow-up PRs.
It isn't implicit it depends on the flags pass https://peps.python.org/pep-0590/. Take care to set the flags to suit your buffer size. |
Yeah I recall the flag, but I just blanket put +1 extra space for everything for simplicity's sake. The failure is here https://buildbot.python.org/all/#/builders/1364/builds/143. |
|
Nevermind, I'll change it so it checks for the flags now. Thanks PY_VECTORCALL_ARGUMENTS_OFFSET! |
|
!buildbot x86-64 MacOS Intel ASAN NoGIL PR |
|
🤖 New build scheduled with the buildbot fleet by @Fidget-Spinner for commit ff41e0c 🤖 The command will test the builders whose names match following regular expression: The builders matched are:
|
…` arguments in configuration command after #118450 (#121083) Signed-off-by: Manjusaka <[email protected]> Co-authored-by: Ken Jin <[email protected]>
…18450) This PR sets up tagged pointers for CPython. The general idea is to create a separate struct _PyStackRef for everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly. Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out. This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it. The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs. Please read Include/internal/pycore_stackref.h for more information! --------- Co-authored-by: Mark Shannon <[email protected]>
…ystats` arguments in configuration command after python#118450 (python#121083) Signed-off-by: Manjusaka <[email protected]> Co-authored-by: Ken Jin <[email protected]>
…18450) This PR sets up tagged pointers for CPython. The general idea is to create a separate struct _PyStackRef for everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly. Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out. This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it. The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs. Please read Include/internal/pycore_stackref.h for more information! --------- Co-authored-by: Mark Shannon <[email protected]>
…ystats` arguments in configuration command after python#118450 (python#121083) Signed-off-by: Manjusaka <[email protected]> Co-authored-by: Ken Jin <[email protected]>
…18450) This PR sets up tagged pointers for CPython. The general idea is to create a separate struct _PyStackRef for everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly. Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out. This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it. The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs. Please read Include/internal/pycore_stackref.h for more information! --------- Co-authored-by: Mark Shannon <[email protected]>
…ystats` arguments in configuration command after python#118450 (python#121083) Signed-off-by: Manjusaka <[email protected]> Co-authored-by: Ken Jin <[email protected]>
This PR sets up tagged pointers for CPython.
The general idea is to create a separate struct
_PyStackReffor everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly.Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out.
This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it.
The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs.
Please read
Include/internal/pycore_stackref.hfor more information!