Repository navigation
Optimize class creation #132042
Description
Activity
I would try if @AA-Turner doesn't pick this up :)
I think the whole concept of slots (the
tp_...slots, not__slots__orPyType_Slot) as an optimization is the root of the problem, they slow down class creation and don't help performance as they complicate the real optimizations that we perform.We should view the
tp_slotsas doing two distinct things:- Specifying the behavior of operations when the
struct _typeobjectis passed toPyType_Ready - A backwards compatible way for C extensions to access operations. Eg.
iter(i)asPy_TYPE(i)->tp_iter
For pure Python objects, all slots can be filled in with a function that does the dynamic lookup, which should be very quick.
Once resolved, we can overwrite the slot with a more direct version.For classes defined by
struct _typeobjectwe can just replace theNULLs with the dynamic lookup function.
For classes defined fromPyType_Specwe fill in the defined slots and then replace theNULLs with the dynamic lookup function.Also, see faster-cpython/ideas#146 (comment)
- Specifying the behavior of operations when the
The bytecode for creating classes is also a bit of a mess. We seem to be creating code objects, just to create functions just to call them, to do things that could easily be done inline.
There is also a fair bit of machinery about finding the metaclass and the base class tuple. We should compute those in the interpreter as pass them into the class creation machinery.
E.g given a
BUILD_CLASSinstruction that expectsname, meta, bases, dictforclass C: ...we get:LOAD_CONST "C" LOAD_CONST object LOAD_CONST () // create method dictionary BUILD_CLASSFor
class C(D): ...we get:LOAD_CONST "C" LOAD_NAME "D" COPY 1 LOAD_ATTR __class__ # Get the metaclass SWAP 2 BUILD_TUPLE 1 // create method dictionary COPY 2 LOAD_ATTR "__prepare__" SWAP 2 CALL 1 # meta.__prepare__(method_dict)For multiple inheritance
LOAD_ATTR __class__becomesCALL_INSTRINSIC 2 calculate_metaclassand
ifmetaclassis explicit,metaclass=expr, thenCOPY 1; LOAD_ATTR __class__becomes justexpr.I'm missing the code for setting
__orig_bases__and__class__, but I think those come afterBUILD_CLASS.A lot of time is spent updating slot definitions, i.e. filling in all of the tp_, nb_, etc. functions in the C struct for the type. We do this by iterating over all the slots, then looking up the function name (e.g., add) in the MRO and placing it in the slot for this class.
Previous attempt in 2017: #76527
I have added some tests results - #132156 (comment)
I want to dig deeper, but if you are interested in some numbers - please take a look. And maybe stop me from further research.Ok, I got rid of
resolve_slotdupsand believe it is ready for review. Please take a look.- added a commit that references this issue
on Apr 17, 2025 - added 5 commits that reference this issue
on Apr 18, 2025 6 remaining items
@JelleZijlstra , do you mind pasting your microbenchmark script
- added 7 commits that reference this issue
on Jun 24, 2025 - added a commit that references this issue
on Mar 12, 2026
Currently, creating an empty class is about 70x slower than creating an empty function in my profiling. Classes are much more complex and it makes sense that they're slower to create, but 70x feels excessive. (Related: #118761.)
I ran some profiling on my Mac with a sample script that just made empty classes in a loop:
A few things stood out:
tp_*,nb_*, etc. functions in the C struct for the type. We do this by iterating over all the slots, then looking up the function name (e.g.,__add__) in the MRO and placing it in the slot for this class.__add__is both nb_add and sq_concat), and does that by iterating over all the slotdefs and finding other slots with the same name. It does that using some scratch space in the interpreter state, which seems not thread-safe. I feel we could precompute the data instead, so we don't have to figure it out at runtime. For example, the slotdef struct could grow a new member to indicate whether or not the name is unique.Most types will define very few of these slots, so it makes sense to try to look for an approach that does less work for slots without changes. I think something like this should work:
__dict__. For those slots only, perform an update.This should make it possible to make class creation something like 2x faster. I haven't started working on implementing this and I may not have time to do it; if you see this and are interested, feel free to pick it up!
Linked PRs