Repository navigation
Module definition abstraction #9
Description
Python modules rely on a static definition of a module. To avoid needing to have a static, threadsafe PyObject, I propose we use PyModExport, which is the new way of defining modules, and does not rely on the safety of a static, threadsafe PyObject.
An example of a module definition, taken from the above PEP:
static PyModuleDef_Slot examplemodule_slots[] = {
{Py_mod_name, "examplemodule"},
{Py_mod_doc, (char*)examplemodule_doc},
{Py_mod_methods, examplemodule_methods},
{Py_mod_state_size, (void*)sizeof(examplemodule_state)},
{Py_mod_exec, (void*)examplemodule_exec},
{0}
};
PyMODEXPORT_FUNC
PyModExport_examplemodule(void)
{
return examplemodule_slots;
}PyMODEXPORT_FUNC is equivalent to *mut PyModuleDef_Slot.
One design I have in mind is something along the lines of:
pub enum PyModuleDefSlot {
Name(&'static CStr),
Doc(&'static CStr),
MultipleInterpreters(*mut c_void),
GIL(*mut c_void),
StateSize(usize),
// ... etc.
}
pub struct PyModuleDef<const N: usize>(UnsafeCell<[ffi::PyModuleDef_Slot; N]>);
impl<const N: usize> PyModuleDef<N> {
pub const fn new(slots: [PyModuleDefSlot; N]) -> Self {
todo!()
}
pub fn as_mut_ptr(&'static self) -> *mut PyModuleDef_Slot {
self.0.get() as *mut PyModuleDef_Slot
}
}
unsafe impl<const N: usize> Sync for PyModuleDef<N> {}
static MODULE: PyModuleDef<5> = PyModule::new([
PyModuleDefSlot::Name(c"mymodule"),
PyModuleDefSlot::Doc(c"An example module for a proof of concept"),
// etc.
]);
#[unsafe(no_mangle)]
unsafe extern "C" fn PyModExport_mymodule() -> *mut PyModuleDef_Slot {
MODULE.as_mut_ptr()
}I think this isn't currently sound because we probably need a thread state-aware lock to prevent races on MODULE.as_mut_ptr()?
We will likely also want a design for specifying methods that are part of the module, which are listed in a static array similar to the PyModExport static.
Activity
davidhewitt commentedon Dec 12, 2025
Given the
PyModExportAPI is brand new, can we ask it to export as*const PyModuleDef_Slotinstead? Immutable static data is generally much nicer and it might be the case that it's mutable only because that's less work in C.cc @encukou
encukou commentedon Dec 12, 2025
Per the docs, “This array must remain valid and constant until interpreter shutdown.”
Can you treat it as immutable?
davidhewitt commentedon Dec 12, 2025
We can pretend and cast the constness away, I guess. Is there a reason you didn't define the API to return
const PyModuleDef_Slot *?encukou commentedon Dec 12, 2025
In C,
constis such a weak annotation that I don't really think about it...ngoldbaum commentedon Dec 12, 2025
Unless someone beats me to it, I’ll be implementing PEP 793 support in PyO3 sometime early next year to enable support for the new stable ABI. PyO3/pyo3#5644
emmatyping commentedon Dec 12, 2025
Yeah, I think this is the best solution.
Since the contents will be immutable/constant, we also don't need a lock to be sound.