Refleaks on free-threaded builds

@ericsnowcurrently

Bug report

Bug description:

The nogil refleak buildbots have started failing for main and 3.14 with leaks in test_sys, test_capi and a few others.

See e.g. https://buildbot.python.org/#/builders/1714/builds/90

I bisected this to 09e72cf, cc @ericsnowcurrently

CPython versions tested on:

CPython main branch, 3.14

Operating systems tested on:

Linux

Linked PRs

gh-134557: Revert "gh-132775: Use _PyCode GetScriptXIData()" #134599
[3.14] gh-134557: Revert "[3.14] gh-132775: Use _PyCode GetScriptXIData()" #134600
gh-134557: Suppress immortalization in _PyCode_GetScriptXIData under free-threading #134686
[3.14] gh-134557: Suppress immortalization in _PyCode_GetScriptXIData under free-threading (gh-134686) #134738

Yeah, I'll be home soon and will take a look.

Any ideas on why there would be leaks only on free-threaded builds?

Also, FWIW, the leaks may actually originate in gh-133482.

Hm, I don't see any obvious leaks on the PRs. I wouldn't be surprised if it only revealed an existing leak on free-threading.

@ericsnowcurrently

Ah, @ericsnowcurrently, this looks like a leak:

cpython/Modules/_interpretersmodule.c

Line 233 in 77eade3

PyObject *res = PyMemoryView_FromObject(obj);

obj needs to decref'd on the successful path. I'm not sure if that's the cause of the buildbots failures, though. No idea why it would only fail under free-threading (maybe it's because the object structure is bigger?)

I tried adding a decref after that but the tests still show a refleak.

FWIW, the results have been consistent for me across runs, so it doesn't seem like a race is involved.

It may be due to interned strings. The only refleak-failing test in test_sys is test_subinterp_intern_dynamically_allocated.

(Thus this is more likely to be an existing bug than one introduced yesterday.)

While reviewing the new code, I saw that

cpython/Modules/_interpretersmodule.c

Lines 877 to 885 in 393773a

    
           PyObject *exc = _PyErr_GetRaisedException(tstate); 
        
           PyObject *cause = PyException_GetCause(exc); 
        
           if (cause != NULL) { 
        
               Py_DECREF(exc); 
        
               exc = cause; 
        
           } 
        
           else { 
        
               assert(PyException_GetContext(exc) == NULL); 
        
           }

should use Py_SETREF instead. I don't think it's the cause of the leak though

In test_capi (test_capi.test_misc), the leak is manifesting exclusively in test_isolated_subinterpreter (specifically the first two subtests).

Yeah, I'm getting [0 refs, 0 blocks] on both FT or GIL-ful builds.

_PyCode_Init() -> intern_one_constant(), which interns the float timeout:

cpython/Objects/codeobject.c

Lines 3088 to 3095 in 7b1010a

    
           #ifdef Py_REF_DEBUG 
        
                   Py_ssize_t refcnt = Py_REFCNT(op); 
        
                   if (refcnt != 1) { 
        
                       // Adjust the reftotal to account for the fact that we only 
        
                       // restore a single reference in _PyCode_Fini. 
        
                       _Py_AddRefTotal(_PyThreadState_GET(), -(refcnt - 1)); 
        
                   } 
        
           #endif

_PyCode_Fini():

cpython/Objects/codeobject.c

Lines 3245 to 3252 in 7b1010a

    
           _PyCode_Fini(PyInterpreterState *interp) 
        
           { 
        
           #ifdef Py_GIL_DISABLED 
        
               // Free interned constants 
        
               struct _py_code_state *state = &interp->code_state; 
        
               if (state->constants) { 
        
                   _Py_hashtable_foreach(state->constants, &clear_containers, NULL); 
        
                   _Py_hashtable_destroy(state->constants);

_interpreters.run_string() -> _PyXI_Exit() -> _exit_session():

cpython/Python/crossinterp.c

Lines 2345 to 2351 in 7b1010a

    
           if (session->prev_tstate != session->init_tstate) { 
        
               assert(session->own_init_tstate); 
        
               session->own_init_tstate = 0; 
        
               PyThreadState_Clear(tstate); 
        
               PyThreadState_Swap(session->prev_tstate); 
        
               PyThreadState_Delete(tstate); 
        
           }

_PyCode_Fini() seems to follow _PyXI_Exit(), which would affect the stat?
(Unlike main, 09e72cf runs _PyCode_Init() and _PyCode_Fini() on different threads.)

@neonene

Thanks for tracking down the leak, @neonene! That's some impressive sleuthing.

Revert "[3.14] pythongh-134557: Revert "[3.14] pythongh-132775: Use _…

gh-134557: Suppress immortalization in _PyCode_GetScriptXIData under …

pythongh-134557: Suppress immortalization in _PyCode_GetScriptXIData …

[3.14] gh-134557: Suppress immortalization in _PyCode_GetScriptXIData…

pythongh-134557: Revert "pythongh-132775: Use _PyCode GetScriptXIData…

pythongh-134557: Suppress immortalization in _PyCode_GetScriptXIData …

pythongh-134557: Revert "pythongh-132775: Use _PyCode GetScriptXIData…

pythongh-134557: Suppress immortalization in _PyCode_GetScriptXIData …

pythongh-134557: Revert "pythongh-132775: Use _PyCode GetScriptXIData…

pythongh-134557: Suppress immortalization in _PyCode_GetScriptXIData …

Navigation Menu

Repository navigation

Refleaks on free-threaded builds #134557

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

9 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	PyObject *exc = _PyErr_GetRaisedException(tstate);
	PyObject *cause = PyException_GetCause(exc);
	if (cause != NULL) {
	Py_DECREF(exc);
	exc = cause;
	}
	else {
	assert(PyException_GetContext(exc) == NULL);
	}

	#ifdef Py_REF_DEBUG
	Py_ssize_t refcnt = Py_REFCNT(op);
	if (refcnt != 1) {
	// Adjust the reftotal to account for the fact that we only
	// restore a single reference in _PyCode_Fini.
	_Py_AddRefTotal(_PyThreadState_GET(), -(refcnt - 1));
	}
	#endif

	_PyCode_Fini(PyInterpreterState *interp)
	{
	#ifdef Py_GIL_DISABLED
	// Free interned constants
	struct _py_code_state *state = &interp->code_state;
	if (state->constants) {
	_Py_hashtable_foreach(state->constants, &clear_containers, NULL);
	_Py_hashtable_destroy(state->constants);

	if (session->prev_tstate != session->init_tstate) {
	assert(session->own_init_tstate);
	session->own_init_tstate = 0;
	PyThreadState_Clear(tstate);
	PyThreadState_Swap(session->prev_tstate);
	PyThreadState_Delete(tstate);
	}

Refleaks on free-threaded builds #134557

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

ericsnowcurrently commented on May 23, 2025

ericsnowcurrently commented on May 23, 2025

ZeroIntensity commented on May 23, 2025

ZeroIntensity commented on May 23, 2025

emmatyping commented on May 23, 2025

ericsnowcurrently commented on May 23, 2025

ericsnowcurrently commented on May 23, 2025

picnixz commented on May 23, 2025

ericsnowcurrently commented on May 23, 2025

9 remaining items

ZeroIntensity commented on May 24, 2025

neonene commented on May 24, 2025

ericsnowcurrently commented on May 26, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions