Repository navigation
test_capi fails if run with --forever argument #105273
Description
Tried on current main:
Fails on test_capi/test_misc/TestOptimizerAPI.test_counter_optimizer
./python -m test -v test_capi -m test_misc -m TestOptimizerAPI --forever
... many lines
======================================================================
FAIL: test_counter_optimizer (test.test_capi.test_misc.TestOptimizerAPI.test_counter_optimizer)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Users\KIRILL-1\CLionProjects\cpython\Lib\test\test_capi\test_misc.py", line 1929, in test_counter_optimizer
self.assertEqual(opt.get_count(), 1000)
AssertionError: 0 != 1000
----------------------------------------------------------------------
Ran 168 tests in 6.590s
FAILED (failures=1, skipped=5)
test test_capi failed
test_capi failed (1 failure)
== Tests result: FAILURE ==
1 test OK.
1 test failed:
test_capi
Total duration: 13.2 sec
Tests result: FAILURE
Activity
Eclips4 commentedon Jun 3, 2023
Seems to be introduced in #105100
cc @markshannon
AlexWaygood commentedon Jun 19, 2023
Fixed by #105683
sunmy2019 commentedon Jun 23, 2023
I am testing heads/main: 6a80664 and the issue happens again.
markshannon commentedon Jun 23, 2023
Unfortunately not.
gvanrossum commentedon Jun 27, 2023
Just got hit by this (it makes the refleak buildbots fail in #105924).
I've got a hunch the problem is in deopt_code() (in codeobject.c), but I can't quite nail it. I put a printf() there when opcode == ENTER_EXECUTOR, and in the loop in the test which executes five times, it prints only four times?! Then the second time the whole test is re-run, it prints never at all (but the test fails after the first time through the loop). It appears the first iteration (out of five in the test --
for repeat in range(5)) doesn't cause any executor to be cleared when clear_executors() setsfunc.__code__ = func.__code__.replace(). But the first iteration clearly has an executor, because the assert passes (the first time the whole test is run).Not sure where this really points, but the logic in deopt_code() is subtle (it gets the opcode from
_Py_GetBaseOpcode(code, i), then patchesinstructions[i]). I get lost when I try to understand_Py_GetBaseOpcode().gvanrossum commentedon Jun 28, 2023
This was now fixed by @vstinner in #106171. His fix is to create a whole new function object (by using
exec). This is needed because the original code object is kept alive by the parent code object -- and the only way to clear a code object'sco_executorsarray is to cause the code object to be finalized.(I tried putting the assignment to
func.__code__in front of theyield, but it didn't seem to work.)Arguably we ought to have an explicit API for this, but that can wait.