Type object's ob_type does not get set when tp_bases is set before PyType_Ready

Bug report

This came up while testing numpy on 3.12. A bug report has been filed on numpy/numpy#23766. It's happening due to #103912, which introduced an additional check for tp_bases not being NULL in type_ready_set_bases, which is called from PyType_Ready.

numpy sets tp_bases manually before calling PyType_Ready, which means that the afore-mentioned check succeeds, and so, the line that sets ob_type does not get executed (it used to before #103912), which leads to a segmentation fault later on, when trying to set mro.

This looks like a bug, but I'm not sure whether that's expected and numpy should be adjusted. If the latter is true, should a note be added in the What's new document?

Linked PRs

@ericsnowcurrently

cc @ericsnowcurrently as the author of #103912.

Verified that the following diff solves this issue, and the test suite passes. (Still not sure whether it has any unintended side-effects though)

diff --git a/Objects/typeobject.c b/Objects/typeobject.c
index 2fbcafe91a..495e123f8f 100644
--- a/Objects/typeobject.c
+++ b/Objects/typeobject.c
@@ -6990,10 +6990,6 @@ type_ready_pre_checks(PyTypeObject *type)
 static int
 type_ready_set_bases(PyTypeObject *type)
 {
-    if (lookup_tp_bases(type) != NULL) {
-        return 0;
-    }
-
     /* Initialize tp_base (defaults to BaseObject unless that's us) */
     PyTypeObject *base = type->tp_base;
     if (base == NULL && type != &PyBaseObject_Type) {

I should have a fix up shortly.

@lysnikolaou

@lysnikolaou, please verify that gh-105122 fixes the problem. If not, what would be a good way to reproduce the problem?

gh-104614: Make Sure ob_type is Always Set Correctly by PyType_Ready() (

pythongh-104614: Make Sure ob_type is Always Set Correctly by PyType_…

[3.12] gh-104614: Make Sure ob_type is Always Set Correctly by PyType…

main and 3.12 should be good now.

pythongh-104614: Fix potential ref. leak in _testcapimodule/get_basic…

gh-104614: Fix potential ref. leak in _testcapimodule/get_basic_stati…

pythongh-104614: Fix potential ref. leak in _testcapimodule/get_basic…

[3.12] gh-104614: Fix potential ref. leak in _testcapimodule/get_basi…

@rwgk

Thanks for the fix. This currently crashes multiple-inheritance code generated by both Cython and pybind11 (@rwgk), so this is a blocker for us (and at least some of our users) testing Py3.12b1. The bug is also difficult to work around, so I'd rather wait for Py3.12b2.

Exclude crashing tests in Python 3.12.0b1.

@scoder

@scoder, do you mean the issue is fixed for you with the patch I landed, or that the patch caused a new problem, or that the issue still isn't fixed, or something else? I think you meant the first one, but I want to be sure. 😄

This fix resolves the issue, so the next beta release should work again.

Thanks for clarifying. It's a relief to hear that. 😄

per https://www.riverbankcomputing.com/pipermail/pyqt/2023-June/045325.html There may still be issues with PyQt6 / sip .

I still see this segfault with current CPython main and it is reported that the failure bisects back to #103912 as well.

I'm the person who posted that email - the issue currently is that:

import sys
from PyQt6.QtWidgets import QWidget, QApplication

class Mixin:
    pass

class Command(Mixin, QWidget):
    pass

app = QApplication(sys.argv)
cmd = Command()
cmd.show()
app.exec()

Segfaults in:

#0  _PyDict_Next (op=0x0, ppos=0x7fffffffbcd0, pkey=0x7fffffffbcd8, pvalue=0x7fffffffbce0, phash=phash@entry=0x0) at Objects/dictobject.c:2114
#1  0x000055555572a2d9 in PyDict_Next (op=<optimized out>, ppos=<optimized out>, pkey=<optimized out>, pvalue=<optimized out>) at Objects/dictobject.c:2189
#2  0x00007ffff2436499 in trawl_hierarchy(_typeobject*, qpycore_metaobject*, QMetaObjectBuilder&, QList<EnumFlag>&, QList<_qpycore_pyqtSignal const*>&, QMap<unsigned int, std::pair<_object*, _object*> >&) ()
   from /home/florian/proj/qutebrowser/.venv/lib/python3.13/site-packages/PyQt6/QtCore.abi3.so
#3  0x00007ffff2436cc7 in trawl_hierarchy(_typeobject*, qpycore_metaobject*, QMetaObjectBuilder&, QList<EnumFlag>&, QList<_qpycore_pyqtSignal const*>&, QMap<unsigned int, std::pair<_object*, _object*> >&) ()
   from /home/florian/proj/qutebrowser/.venv/lib/python3.13/site-packages/PyQt6/QtCore.abi3.so
#4  0x00007ffff2436cc7 in trawl_hierarchy(_typeobject*, qpycore_metaobject*, QMetaObjectBuilder&, QList<EnumFlag>&, QList<_qpycore_pyqtSignal const*>&, QMap<unsigned int, std::pair<_object*, _object*> >&) ()
   from /home/florian/proj/qutebrowser/.venv/lib/python3.13/site-packages/PyQt6/QtCore.abi3.so
#5  0x00007ffff24357ba in qpycore_get_dynamic_metaobject(_sipWrapperType*) [clone .part.0] () from /home/florian/proj/qutebrowser/.venv/lib/python3.13/site-packages/PyQt6/QtCore.abi3.so
#6  0x00007ffff24363d8 in qpycore_get_qmetaobject(_sipWrapperType*, _sipTypeDef const*) () from /home/florian/proj/qutebrowser/.venv/lib/python3.13/site-packages/PyQt6/QtCore.abi3.so
#7  0x00007ffff242a4e5 in qpycore_qobject_finalisation(_object*, QObject*, _object*, _object**) () from /home/florian/proj/qutebrowser/.venv/lib/python3.13/site-packages/PyQt6/QtCore.abi3.so
#8  0x00007ffff74f0569 in sipSimpleWrapper_init (self=0x7ffff7652840, args=(), kwds=0x0) at sip_core.c:8950
#9  0x000055555575906b in type_call (type=<optimized out>, type@entry=0x555555ea2470, args=args@entry=(), kwds=kwds@entry=0x0) at Objects/typeobject.c:1671
#10 0x00005555556eb3b2 in _PyObject_MakeTpCall (tstate=tstate@entry=0x555555c76688 <_PyRuntime+475432>, callable=callable@entry=<PyQt6.sip.wrappertype at remote 0x555555ea2470>, args=args@entry=0x7ffff7f7f078, nargs=<optimized out>, 
    keywords=keywords@entry=0x0) at Objects/call.c:240
[...]

And indeed I could bisect this to de64e75. The main
branch fails in the same way.

Happy to open a new issue, but I'm first trying to see if the PyQt author can perhaps
help boil this down to a more minimal example not requiring PyQt.

@The-Compiler

@The-Compiler, this is a separate issue, related to tp_dict rather than tp_bases and ob_type. It would be worth opening a new issue for this.

In the meantime, is the failure with a builtin static type or a user-defined type? Was PyType_Ready() called on that type already? Do you have the full stack trace?

Presumably the crashing code is trying to use the dict returned by sipPyTypeDict() and I'm guessing that that function basically returns pytype->tp_dict and that the pytype in question is a static builtin type. If that's the case then the failure makes sense since tp_dict for static builtin types is now NULL. In 3.12 we added an internal getter function (_PyType_GetDict() via Py_BUILD_CORE) but not a public one.

This sounds like gh-105227. If so, let's continue the conversation there. If not then let's create a new issue.

Either way, I'm closing this issue again.

Navigation Menu

Repository navigation

Type object's ob_type does not get set when tp_bases is set before PyType_Ready #104614

Bug report

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Type object's ob_type does not get set when tp_bases is set before PyType_Ready #104614

Description

Bug report

Linked PRs

Activity

lysnikolaou commented on May 23, 2023

lysnikolaou commented on May 23, 2023

ericsnowcurrently commented on May 30, 2023

ericsnowcurrently commented on May 31, 2023

ericsnowcurrently commented on Jun 1, 2023

scoder commented on Jun 4, 2023

ericsnowcurrently commented on Jun 5, 2023

scoder commented on Jun 5, 2023

ericsnowcurrently commented on Jun 5, 2023

tacaswell commented on Jun 9, 2023

The-Compiler commented on Jun 9, 2023

ericsnowcurrently commented on Jun 13, 2023

ericsnowcurrently commented on Jun 13, 2023

ericsnowcurrently commented on Jun 13, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions