Repository navigation
Crash when inspecting frame of generator #94262
Description
Activity
This is tricky.
The fundamental problem, IMO, is that almost any memory allocation can run arbitrary code. This makes it really difficult to reason about the state of the VM.
In the short term, we should handle the case where
frame->frame_obj != NULLwhen creating a generator, but there are bound other issues, as the generator and frame object are may not be an introspectable state.In the longer term, I'd like to only run the GC, and/or finalizers, when the VM is a known state; as we do with interrupts.
I'm concerned that this shows up from an interrupt. It shouldn't.
The fundamental problem, IMO, is that almost any memory allocation can run arbitrary code. This makes it really difficult to reason about the state of the VM.
I agree, not only the state of VM but also any other object state can be mutated which in some can lead to memory leak or worse crash.
I'm concerned that this shows up from an interrupt. It shouldn't.
It was really hard to reproduce the error, my current theory is that if an interrupt happens while executing
__del__of an object and the interpreter creates a traceback to print, it can create a frame object for a generator and it causes a hard crash while the exception is propagated to the generator.I was correct about my theory, the following code sample demonstrates how
KeyboardInterruptcan lead to crash:import gc gc.set_threshold(1, 0, 0) class Sneaky: def __del__(self): raise KeyboardInterrupt def gen(): yield 1 sneaky = Sneaky() sneaky._s = Sneaky() sneaky._s._s = sneaky del sneaky g = gen()
Crash:
Exception ignored in: <function Sneaky.__del__ at 0x7fbfa49220a0> Traceback (most recent call last): File "/workspaces/cpython/main.py", line 6, in __del__ raise KeyboardInterrupt ^^^^^^^^^^^^^^^^^^^^^^^ KeyboardInterrupt: Exception ignored in: <function Sneaky.__del__ at 0x7fbfa49220a0> Traceback (most recent call last): File "/workspaces/cpython/main.py", line 6, in __del__ raise KeyboardInterrupt ^^^^^^^^^^^^^^^^^^^^^^^ KeyboardInterrupt: python: Python/ceval.c:5477: _PyEval_EvalFrameDefault: Assertion `frame->frame_obj == NULL' failed.
Reacted by Michael DroettboomThanks @kumaraditya303 that's really helpful.
Turns out that the fix isn't that hard.
The problem isn't really the GC behavior* but that partially constructed frames are visible to Python and C extensions.
We fix this by not creating frame objects for partially constructed frames.*The behavior of the GC and finalization is a problem, but not the problem here.
- added a commit that references this issue
on Jul 1, 2022 - added a commit that references this issue
on Jul 4, 2022 Thanks, @markshannon and @kumaraditya303 for the work! 🎉
Reacted by Kumar Aditya
Crash report
This was discovered in an
asyncioprogram when interrupted with CTRL - C.Minimal Reproducer:
Error messages
Your environment
cc @markshannon @pablogsal