Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-76785: Clean Up Interpreter ID Conversions #117048

Merged
merged 7 commits into from
Mar 21, 2024
Merged

gh-76785: Clean Up Interpreter ID Conversions #117048

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d299396
Clean up interpreter ID conversions.
ericsnowcurrently Mar 19, 2024
9869010
Add a section specifically about interpreter ID.
ericsnowcurrently Mar 20, 2024
76db436
Add _PyInterpreterID_GetID().
ericsnowcurrently Mar 20, 2024
a10fe2a
_PyInterpreterID_ObjectToID() -> _PyInterpreterState_ObjectToID()
ericsnowcurrently Mar 20, 2024
7154bb6
PyInterpreterID_LookUp() -> PyInterpreterState_LookUpIDObject()
ericsnowcurrently Mar 20, 2024
6f9e5ec
Move functions to the right place.
ericsnowcurrently Mar 20, 2024
1d3e30f
PyInterpreterState_LookUpIDObject() -> _PyInterpreterState_LookUpIDOb…
ericsnowcurrently Mar 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Include/cpython/interpreteridobject.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,7 @@ PyAPI_DATA(PyTypeObject) PyInterpreterID_Type;

PyAPI_FUNC(PyObject *) PyInterpreterID_New(int64_t);
PyAPI_FUNC(PyObject *) PyInterpreterState_GetIDObject(PyInterpreterState *);
PyAPI_FUNC(PyInterpreterState *) PyInterpreterID_LookUp(PyObject *);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that in 3.12 this was a private API (leading _) but in 3.13 it lost the _. It looks like you're going back to private. What about the handful of definitions above this line? Are any of them meant to be public?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question. They needn't be public at this point and it turns out interpreteridobject.h was never added to Python.h, so there shouldn't be a problem moving them. I'll do that here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, I'm going to remove the InterpreterID type, including this header file, in a separate PR. (I made some changes a few months ago that made the class unnecessary. It can be removed with a couple small tweaks.)


#ifdef Py_BUILD_CORE
extern int64_t _PyInterpreterID_GetID(PyObject *);
#endif
3 changes: 3 additions & 0 deletions Include/internal/pycore_interp.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -295,8 +295,11 @@ _PyInterpreterState_SetFinalizing(PyInterpreterState *interp, PyThreadState *tst
}


extern int64_t _PyInterpreterState_ObjectToID(PyObject *);

// Export for the _xxinterpchannels module.
PyAPI_FUNC(PyInterpreterState *) _PyInterpreterState_LookUpID(int64_t);
PyAPI_FUNC(PyInterpreterState *) PyInterpreterState_LookUpIDObject(PyObject *);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this have a leading _ as well, like the others below?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'll fix that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


PyAPI_FUNC(int) _PyInterpreterState_IDInitref(PyInterpreterState *);
PyAPI_FUNC(int) _PyInterpreterState_IDIncref(PyInterpreterState *);
Expand Down
4 changes: 2 additions & 2 deletions Lib/test/test_capi/test_misc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2185,7 +2185,7 @@ def test_equality(self):

def test_linked_lifecycle(self):
id1 = _interpreters.create()
_testcapi.unlink_interpreter_refcount(id1)
_testinternalcapi.unlink_interpreter_refcount(id1)
self.assertEqual(
_testinternalcapi.get_interpreter_refcount(id1),
0)
Expand All @@ -2201,7 +2201,7 @@ def test_linked_lifecycle(self):
_testinternalcapi.get_interpreter_refcount(id1),
0)

_testcapi.link_interpreter_refcount(id1)
_testinternalcapi.link_interpreter_refcount(id1)
self.assertEqual(
_testinternalcapi.get_interpreter_refcount(id1),
0)
Expand Down
26 changes: 0 additions & 26 deletions Modules/_testcapimodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1455,30 +1455,6 @@ get_interpreterid_type(PyObject *self, PyObject *Py_UNUSED(ignored))
return Py_NewRef(&PyInterpreterID_Type);
}

static PyObject *
link_interpreter_refcount(PyObject *self, PyObject *idobj)
{
PyInterpreterState *interp = PyInterpreterID_LookUp(idobj);
if (interp == NULL) {
assert(PyErr_Occurred());
return NULL;
}
_PyInterpreterState_RequireIDRef(interp, 1);
Py_RETURN_NONE;
}

static PyObject *
unlink_interpreter_refcount(PyObject *self, PyObject *idobj)
{
PyInterpreterState *interp = PyInterpreterID_LookUp(idobj);
if (interp == NULL) {
assert(PyErr_Occurred());
return NULL;
}
_PyInterpreterState_RequireIDRef(interp, 0);
Py_RETURN_NONE;
}

static PyMethodDef ml;

static PyObject *
Expand Down Expand Up @@ -3297,8 +3273,6 @@ static PyMethodDef TestMethods[] = {
{"test_current_tstate_matches", test_current_tstate_matches, METH_NOARGS},
{"run_in_subinterp", run_in_subinterp, METH_VARARGS},
{"get_interpreterid_type", get_interpreterid_type, METH_NOARGS},
{"link_interpreter_refcount", link_interpreter_refcount, METH_O},
{"unlink_interpreter_refcount", unlink_interpreter_refcount, METH_O},
{"create_cfunction", create_cfunction, METH_NOARGS},
{"call_in_temporary_c_thread", call_in_temporary_c_thread, METH_VARARGS,
PyDoc_STR("set_error_class(error_class) -> None")},
Expand Down
32 changes: 28 additions & 4 deletions Modules/_testinternalcapi.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,6 @@
#include "pycore_pyerrors.h" // _PyErr_ChainExceptions1()
#include "pycore_pystate.h" // _PyThreadState_GET()

#include "interpreteridobject.h" // PyInterpreterID_LookUp()

#include "clinic/_testinternalcapi.c.h"

// Include test definitions from _testinternalcapi/
Expand Down Expand Up @@ -1112,7 +1110,7 @@ pending_identify(PyObject *self, PyObject *args)
if (!PyArg_ParseTuple(args, "O:pending_identify", &interpid)) {
return NULL;
}
PyInterpreterState *interp = PyInterpreterID_LookUp(interpid);
PyInterpreterState *interp = PyInterpreterState_LookUpIDObject(interpid);
if (interp == NULL) {
if (!PyErr_Occurred()) {
PyErr_SetString(PyExc_ValueError, "interpreter not found");
Expand Down Expand Up @@ -1480,13 +1478,37 @@ run_in_subinterp_with_config(PyObject *self, PyObject *args, PyObject *kwargs)
static PyObject *
get_interpreter_refcount(PyObject *self, PyObject *idobj)
{
PyInterpreterState *interp = PyInterpreterID_LookUp(idobj);
PyInterpreterState *interp = PyInterpreterState_LookUpIDObject(idobj);
if (interp == NULL) {
return NULL;
}
return PyLong_FromLongLong(interp->id_refcount);
}

static PyObject *
link_interpreter_refcount(PyObject *self, PyObject *idobj)
{
PyInterpreterState *interp = PyInterpreterState_LookUpIDObject(idobj);
if (interp == NULL) {
assert(PyErr_Occurred());
return NULL;
}
_PyInterpreterState_RequireIDRef(interp, 1);
Py_RETURN_NONE;
}

static PyObject *
unlink_interpreter_refcount(PyObject *self, PyObject *idobj)
{
PyInterpreterState *interp = PyInterpreterState_LookUpIDObject(idobj);
if (interp == NULL) {
assert(PyErr_Occurred());
return NULL;
}
_PyInterpreterState_RequireIDRef(interp, 0);
Py_RETURN_NONE;
}


static void
_xid_capsule_destructor(PyObject *capsule)
Expand Down Expand Up @@ -1728,6 +1750,8 @@ static PyMethodDef module_functions[] = {
_PyCFunction_CAST(run_in_subinterp_with_config),
METH_VARARGS | METH_KEYWORDS},
{"get_interpreter_refcount", get_interpreter_refcount, METH_O},
{"link_interpreter_refcount", link_interpreter_refcount, METH_O},
{"unlink_interpreter_refcount", unlink_interpreter_refcount, METH_O},
{"compile_perf_trampoline_entry", compile_perf_trampoline_entry, METH_VARARGS},
{"perf_trampoline_set_persist_after_fork", perf_trampoline_set_persist_after_fork, METH_VARARGS},
{"get_crossinterp_data", get_crossinterp_data, METH_VARARGS},
Expand Down
82 changes: 4 additions & 78 deletions Modules/_xxsubinterpretersmodule.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,83 +35,8 @@ _get_current_interp(void)
return PyInterpreterState_Get();
}

static int64_t
pylong_to_interpid(PyObject *idobj)
{
assert(PyLong_CheckExact(idobj));

if (_PyLong_IsNegative((PyLongObject *)idobj)) {
PyErr_Format(PyExc_ValueError,
"interpreter ID must be a non-negative int, got %R",
idobj);
return -1;
}

int overflow;
long long id = PyLong_AsLongLongAndOverflow(idobj, &overflow);
if (id == -1) {
if (!overflow) {
assert(PyErr_Occurred());
return -1;
}
assert(!PyErr_Occurred());
// For now, we don't worry about if LLONG_MAX < INT64_MAX.
goto bad_id;
}
#if LLONG_MAX > INT64_MAX
if (id > INT64_MAX) {
goto bad_id;
}
#endif
return (int64_t)id;

bad_id:
PyErr_Format(PyExc_RuntimeError,
"unrecognized interpreter ID %O", idobj);
return -1;
}

static int64_t
convert_interpid_obj(PyObject *arg)
{
int64_t id = -1;
if (_PyIndex_Check(arg)) {
PyObject *idobj = PyNumber_Long(arg);
if (idobj == NULL) {
return -1;
}
id = pylong_to_interpid(idobj);
Py_DECREF(idobj);
if (id < 0) {
return -1;
}
}
else {
PyErr_Format(PyExc_TypeError,
"interpreter ID must be an int, got %.100s",
Py_TYPE(arg)->tp_name);
return -1;
}
return id;
}

static PyInterpreterState *
look_up_interp(PyObject *arg)
{
int64_t id = convert_interpid_obj(arg);
if (id < 0) {
return NULL;
}
return _PyInterpreterState_LookUpID(id);
}

#define look_up_interp PyInterpreterState_LookUpIDObject

static PyObject *
interpid_to_pylong(int64_t id)
{
assert(id < LLONG_MAX);
return PyLong_FromLongLong(id);
}

static PyObject *
get_interpid_obj(PyInterpreterState *interp)
Expand All @@ -123,7 +48,8 @@ get_interpid_obj(PyInterpreterState *interp)
if (id < 0) {
return NULL;
}
return interpid_to_pylong(id);
assert(id < LLONG_MAX);
return PyLong_FromLongLong(id);
}

static PyObject *
Expand Down Expand Up @@ -699,7 +625,7 @@ interp_set___main___attrs(PyObject *self, PyObject *args)
}

// Look up the interpreter.
PyInterpreterState *interp = PyInterpreterID_LookUp(id);
PyInterpreterState *interp = look_up_interp(id);
if (interp == NULL) {
return NULL;
}
Expand Down
66 changes: 23 additions & 43 deletions Objects/interpreteridobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
/* InterpreterID object */

#include "Python.h"
#include "pycore_abstract.h" // _PyIndex_Check()
#include "pycore_interp.h" // _PyInterpreterState_LookUpID()
#include "pycore_interp.h" // _PyInterpreterState_LookUpID()
#include "interpreteridobject.h"


Expand All @@ -11,6 +10,21 @@ typedef struct interpid {
int64_t id;
} interpid;

int64_t
_PyInterpreterID_GetID(PyObject *self)
{
if (!PyObject_TypeCheck(self, &PyInterpreterID_Type)) {
PyErr_Format(PyExc_TypeError,
"expected an InterpreterID, got %R",
self);
return -1;

}
int64_t id = ((interpid *)self)->id;
assert(id >= 0);
return id;
}

static interpid *
newinterpid(PyTypeObject *cls, int64_t id, int force)
{
Expand Down Expand Up @@ -42,43 +56,19 @@ newinterpid(PyTypeObject *cls, int64_t id, int force)
return self;
}

static int
interp_id_converter(PyObject *arg, void *ptr)
{
int64_t id;
if (PyObject_TypeCheck(arg, &PyInterpreterID_Type)) {
id = ((interpid *)arg)->id;
}
else if (_PyIndex_Check(arg)) {
id = PyLong_AsLongLong(arg);
if (id == -1 && PyErr_Occurred()) {
return 0;
}
if (id < 0) {
PyErr_Format(PyExc_ValueError,
"interpreter ID must be a non-negative int, got %R", arg);
return 0;
}
}
else {
PyErr_Format(PyExc_TypeError,
"interpreter ID must be an int, got %.100s",
Py_TYPE(arg)->tp_name);
return 0;
}
*(int64_t *)ptr = id;
return 1;
}

static PyObject *
interpid_new(PyTypeObject *cls, PyObject *args, PyObject *kwds)
{
static char *kwlist[] = {"id", "force", NULL};
int64_t id;
PyObject *idobj;
int force = 0;
if (!PyArg_ParseTupleAndKeywords(args, kwds,
"O&|$p:InterpreterID.__init__", kwlist,
interp_id_converter, &id, &force)) {
"O|$p:InterpreterID.__init__", kwlist,
&idobj, &force)) {
return NULL;
}
int64_t id = _PyInterpreterState_ObjectToID(idobj);
if (id < 0) {
return NULL;
}

Expand Down Expand Up @@ -282,13 +272,3 @@ PyInterpreterState_GetIDObject(PyInterpreterState *interp)
}
return (PyObject *)newinterpid(&PyInterpreterID_Type, id, 0);
}

PyInterpreterState *
PyInterpreterID_LookUp(PyObject *requested_id)
{
int64_t id;
if (!interp_id_converter(requested_id, &id)) {
return NULL;
}
return _PyInterpreterState_LookUpID(id);
}
Loading
Loading