Heap types implemented in C must have Py_TPFLAGS_HAVE_GC and implement tp_traverse

If a heap type does not have Py_TPFLAGS_HAVE_GC or doesn't not implement tp_traverse, the garbage collector may fail to collect the type and most of its instances at Python exit. The problem is worse when sub-interpreters are involved: if a sub-interpreter exits and does not clear all of its memory, the other interpreters will continue running with these leaks.

See issue gh-87138 for the rationale.

I wrote an article on the lock type which had this bug and caused a memory leak: https://vstinner.github.io/subinterpreter-leaks.html

Linked PRs

The following PRs were directly closed because the types are immutable and empty:

The following PRs were wrongly merged so we reverted them:

PRs for reverting the above commits:

pythongh-116946: Emit RuntimeError if a heap does not implement traverse

See also issue gh-116915 "test_capi leaks references" and PR gh-116934 "Make _thread._ThreadHandle support GC" which fix it.

I use the following convention:

When a list item has been addressed, I add a checkbox.
When the behavior has been changed in main, I add ✅.
If nothing needs to be done, I cross the name.

The following items may need to support the GC protocol or be made immutable. Once this is done, this first list must be empty and each of its member will be moved to one of the lists below (may be present in more than one list).

_interpreters.CrossInterpreterBufferView (?)

The following types are true container types that should support the GC protocol and have reference cycles with public documented API only. In particular, the corresponding PRs need to be backported.

The following types are true container types that should support the GC protocol. This list tracks, in addition, their immutability in main (if this is not already the case):

The following types are empty (or their attributes cannot be used to create cycles) and can be made immutable in main. Non-immutable trivial types are left as is in bugfix versions to alleviate the burden on the GC even if it's possible to create reference cycles manually (and require the user to dig into implementation details):

The following types are immutable and empty (or cannot create cycles), hence do not need the GC. If they do implement the GC protocol, they should be simplified.

The following types are immutable but not empty, but really internal.
Visiting them is delegated to someone else:

✅ ~~functools._lru_list_elem~~

Finally, the following types are explicitly required to be missing the GC (testing purposes):

✅ ~~_testcapi.HeapCTypeSetattr~~
✅ ~~_testcapi.HeapCType~~
✅ ~~_testcapi.HeapCTypeSubclass~~
✅ ~~_testcapi.HeapCTypeSubclassWithFinalizer~~ (inherits _testcapi.HeapCTypeSubclass)

The following should require the GC protocol. Currently the tests do not leak, but they could if we were to store the instance as one of its attribute. Nevertheless, since we're only testing them once, I suggest we leave them alone.

_testcapi.HeapCTypeWithDict2
_testcapi.HeapCTypeWithDict
_testcapi.HeapCTypeWithNegativeDict
_testcapi.HeapCTypeWithRelativeDict

The following testing types are also empty or trivial:

The following testing types are non-empty but do not own the references:

✅ ~~_testcapi.HeapCTypeWithRelativeWeakref~~
✅ ~~_testcapi.HeapCTypeWithWeakref2~~
✅ ~~_testcapi.HeapCTypeWithWeakref~~

Backports delayed until https://github.com/python/cpython/issues/138791 is fixed. ↩ ↩²

pythongh-116946: fully implement GC protocol for `lzma` objects

gh-116946: fully implement GC protocol for `bz2` objects (#138266)

pythongh-116946: fully implement GC protocol for `bz2` objects (pytho…

gh-116946: fully implement GC protocol for `lzma` objects (#138288)

gh-116946: fully implement GC protocol for `_hashlib` objects (#138289)

I'm wondering why _thread._ThreadHandle (immutable heaptype) caused ref leaks without GC support in #116915.

That wasn't a real refleak and it occurred because register_at_fork was called, which the refleak checker mistakenly identifies as a leak. This is a known, longstanding issue in CPython that fork handlers cannot be cleared.

See this comment

cpython/Lib/test/test_posix.py

Lines 243 to 245 in ce23eea

    
               os.register_at_fork(before=lambda: None, after_in_child='') 
        
           # We test actual registrations in their own process so as not to 
        
           # pollute this one.  There is no way to unregister for cleanup.

gh-116946: remove unnecessary gc from immutable types (#139073)

Revert "pythongh-116946: remove unnecessary gc from immutable types (p…

gh-116946: partial revert gh-139073 for thread handle type (#139474)

GH-116946: eliminate the need for the GC in the `_thread.lock` and `_…

Revert "pythonGH-116946: eliminate the need for the GC in the `_threa…

GH-116946: revert eliminate the need for the GC in the `_thread.lock`…

pythonGH-116946: eliminate the need for the GC in the `_thread.lock` …

pythonGH-116946: revert eliminate the need for the GC in the `_thread…

I'm in the process of trying to convert a single phase init module (https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git/tree/bindings/python/tests/gpiosim/ext.c) over to multi-phase init in light of PEP 793 looking to soft deprecate PyInit_*.

I'm trying to keep up with the back and forth on Py_TPFLAGS_IMMUTABLETYPE and types created via PyType_FromModuleAndSpec which are Py_TPFLAGS_HEAPTYPE.

In our case, the type does need access to module state, so needs to be a heap type after the transition.

What's not clear is whether I also need to define this as Py_TPFLAGS_HAVE_GC and define tp_traverse. It looks like the standard modules like bz2 (which is probably our closest analogue) has been marked GC, unmarked GC, and maybe still under consideration to be added back #139073 (comment)

If I'm reading #118021 (comment) correctly, if a heap type is created via PyType_FromSpec and thus has no module ref, it may be possible to avoid the GC flag, otherwise if using methods like PyType_FromModuleAndSpec or PyType_FromMetaclass with a module argument, then there may be a need to add the GC flag and a tp_traverse method?

Pep 573 https://peps.python.org/pep-0573/#adding-module-references-to-heap-types:

Usually, creating a class with ht_module set will create a reference cycle involving the class and the module. This is not a problem, as tearing down modules is not a performance-sensitive operation, and module-level functions typically also create reference cycles. The existing “set all module globals to None” code that breaks function cycles through f_globals will also break the new cycles through ht_module.

Have there been any new discoveries or is there any clarification on what's recommended for C extension maintainers? Just looking for some guidance, otherwise I may just punt on converting this until this issue has been resolved.

PR #152310 reimplements the GC protocol for tkapp and tktimertoken, addressing the two problems that led to the gh-138331 revert (gh-138807):

The shutdown crash (gh-138791: fix crash when visiting a non-consumed timer handler #138789): the GC slots use a plain cast instead of the type-checking macro, since the collector can visit a surviving object after module_clear() has reset the global type pointers.
The cancelled-timer regression: a pending timer is kept alive by the Tcl event loop, so it is treated as a GC root (only its callback is traversed) and collecting it no longer cancels a live timer.

Both are covered by regression tests.

pythongh-116946: Implement the GC protocol for _tkinter tkapp and tkt…

gh-116946: Implement the GC protocol for _tkinter tkapp and tktimerto…

@serhiy-storchaka

Have there been any new discoveries or is there any clarification on what's recommended for C extension maintainers?

I think we should create a separate documentation issue. @serhiy-storchaka ?

As currently we have following advice in docs:

Instances of heap types hold a reference to their type. This ensures that the type isn’t destroyed before all its instances are, but may result in reference cycles that need to be broken by the garbage collector.

To avoid memory leaks, instances of heap types must implement the garbage collection protocol. That is, heap types should:

Have the Py_TPFLAGS_HAVE_GC flag.

Define a traverse function using Py_tp_traverse, which visits the type (e.g. using Py_VISIT(Py_TYPE(self))).

If that's actually optional, I believe it worth to be documented. Using GC-enabled immutable heap types come with a noticeable performance loss. Here is an example for the _decimal module: skirpichev#24

Navigation Menu

Repository navigation

Heap types implemented in C must have Py_TPFLAGS_HAVE_GC and implement tp_traverse #116946

Linked PRs

64 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	os.register_at_fork(before=lambda: None, after_in_child='')
	# We test actual registrations in their own process so as not to
	# pollute this one. There is no way to unregister for cleanup.

Heap types implemented in C must have Py_TPFLAGS_HAVE_GC and implement tp_traverse #116946

Description

Linked PRs

Activity

vstinner commented on Mar 18, 2024

vstinner commented on Mar 18, 2024

picnixz commented on Aug 30, 2025

Footnotes

64 remaining items

kumaraditya303 commented on Oct 1, 2025

vfazio commented on Apr 11, 2026

serhiy-storchaka commented on Jun 26, 2026

skirpichev commented on Aug 16, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions