You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Some builtin single phase modules do get loaded into subinterpreters which do not support them #119450
I expect that no single phase module is allowed to be loaded. This works for all self build modules and with most builtin ones, but not all. Some modules i found to be misbehaving are e.g. datetime, tracemalloc.
This can easily cause a crash:
#include<Python.h>#include<assert.h>// make subinterpreter from configPyThreadState*make_sub(PyInterpreterConfig*config)
{
PyThreadState*sub=NULL;
PyStatusstatus=Py_NewInterpreterFromConfig(&sub, config);
if (PyStatus_Exception(status)) {
returnNULL;
}
returnsub;
}
// create an isolated subinterpreter, then load a module with name 'name'voidsub_import(PyThreadState*main, constchar*name)
{
PyEval_AcquireThread(main);
PyInterpreterConfigconfig= {
.use_main_obmalloc=0,
.allow_fork=0,
.allow_exec=0,
.allow_threads=1,
.allow_daemon_threads=0,
.check_multi_interp_extensions=1,
.gil=PyInterpreterConfig_OWN_GIL,
};
PyThreadState*sub=make_sub(&config);
assert(sub);
PyObject*module=PyImport_ImportModule(name);
printf("Loaded: %s\n", module ? "yes" : "no");
Py_XDECREF(module);
Py_EndInterpreter(sub);
}
intmain()
{
Py_InitializeEx(0);
PyThreadState*interp=PyEval_SaveThread();
constchar*module_name="datetime";
sub_import(interp, module_name);
sub_import(interp, module_name); // <- crashPyEval_AcquireThread(interp);
Py_FinalizeEx();
}
This is because the module will get loaded a second time, which will cause python to try and cleanup the old state from the old interpreter inside the new interpreter.
// obmalloc.cvoid_PyObject_Free(void*ctx, void*p) // p will point to an object allocated by pymalloc from the old interpreter here
{
/* PyObject_Free(NULL) has no effect */if (p==NULL) {
return;
}
OMState*state=get_state();
if (UNLIKELY(!pymalloc_free(state, ctx, p))) { // <- this returns false, since the object does not belong to the new interpreter/* pymalloc didn't allocate this address */PyMem_RawFree(p); // <- this will crash, since the pointer was not allocated directly through mallocraw_allocated_blocks--;
}
}
This may be related to #117953 and #104621 but its important to note that this does not happen with the readline module as it correctly refuses to load.
// _tetimemodule.c:6755staticint_datetime_exec(PyObject*module)
{
...
PyObject*d=PyDateTime_DeltaType.tp_dict; // <- this is a global variableDATETIME_ADD_MACRO(d, "resolution", new_delta(0, 0, 1, 0));
Activity
sebsura commentedon May 23, 2024
The deallocation is caused by this:
ericsnowcurrently commentedon Jul 8, 2024
This should be resolved now for 3.13+, as of gh-121060. Sadly, the fix is a bit too complex to backport to 3.12.
sebsura commentedon Jul 9, 2024
Thanks for the heads up. I can now run the sample code i gave above without crashing!