Repository navigation
Do Not Allow Static Objects to be Deallocated #101265
Description
Activity
Possible solutions:
- check address range
- check a per-object flag
(We'd probably hide the check behind one or more internal/static helper functions.)
int _Py_IsStaticObject(PyObject *obj); // or, in the respective Objects/*.c files: int is_static_bytes(PyObject *obj); int is_static_int(PyObject *obj); int is_static_tuple(PyObject *obj); ...
(For efficiency those could also be
static inlineor even macros.)Check Address Range
An "obvious" solution is to check if the pointer is in the address range of known static objects (e.g.
_PyRuntime.static_objects). However, the C standard says that checking if a pointer is in an address range is undefined. We've have to get clever about it.Per Raymond Chen, for architectures with "flat" addressing and where the compiler's pointer-to-integer conversion produces the numeric value of the linear address, we can convert-then-compare for address ranges. So we'd add something to
Include/pyport.hto encapsulate that.For architectures/compilers that don't support that, we'd have to do a linear search for the address.
Check a Per-Object Flag
For every static object we would set a bit somewhere on the object indicating that it is static. Then we could check that.
We already have a check like this for the singletons and for
str.How is it done for
str? Is there a spare bit in the string object?Also I've lost track of the draft PR where Eddie's work is happening. Can someone link to it?
Also I've lost track of the draft PR where Eddie's work is happening. Can someone link to it?
Reacted by Eddie ElizondoDeepfreeze adds the following:
sliceUnless I missed something, deepfreeze does not generates static slice objects.
Reacted by Eric SnowUnless I missed something, deepfreeze does not generates static slice objects.
Correct, there's no slice object in deepfreeze, it's these types:
bytes,code,complex,float,int, tuple, andstrHow is it done for str?
Upstream we have
unicode_is_singletonbut it's not all-inclusive at the moment. The function only compiles in debug mode and it only checks for a certain category of static strings (like the empty string and some single code point unicode objects)In the Immortal PR, I added extra signaling by extending
PyASCIIObject.state.internedfrom 1 bit to 2 bits to include:SSTATE_INTERNED_IMMORTAL. This works for deepfreeze str objects since we call_PyStaticCode_Initwhich then interns these objects, though I don't think we do this forpycore_global_objects .hyetThis gives us two options for strings, either guaranteeing a call to intern all static strings (through the interpreter initialization), or just adding an extra bit in the unicode state, say:
PyASCIIObject.state.static
The runtime currently has many objects that are statically defined. See Include/internal/pycore_global_objects.h. For the singletons, all instances of the type are statically defined. Otherwise only some instances are.
(affected types)
The following types are those that have only some statically defined instances (not counting deep-frozen objects):
intbytesstrtupleDeepfreeze adds the following:
floatcomplexslicecodebytes,int,tuple, &strobjects)The problem is that if
tp_dealloc()for one of the affected types tries to free a static object then it will crash. We've set the refcount for such objects to a really high number to avoid this, but it is still possible. The risk rises a bit for immortal objects (see PEP 683).For all types with static objects, we need to add a check in
tp_dealloc()to either fail or reset the refcount to the really high number. We already have a check like this for the singletons and forstr. For the other types we need to identify if the object is static and respond accordingly. The catch is that identifying that can be expensive, which is problematic for types that are deallocated frequently.