Repository navigation
Improve performance by replacing PyTuple_Pack with PyTuple_FromArray #140009
Description
Feature or enhancement
Proposal:
The PyTuple_Pack method is relatively slow because it is a variadic function. For various cases (in particular tuples of size 1 and 2) we can replace PyTuple_Pack with PyTuple_FromArray (introduced in #111489)
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
Linked PRs
- gh-140009: Improve performance of list_extend_dictitems by using PyTuple_FromArray #140010
- gh-140009: Optimize dict object by replacing PyTuple_Pack with PyTuple_FromArray #144529
- gh-140009: Optimize dict object by replacing PyTuple_Pack with PyTuple_FromArray #144531
- gh-140009: Improve performance of datetime extension module by using PyTuple_FromArray #144532
- gh-140009: Replace PyTuple_Pack with PyTuple_FromArray in _warnings #144760
- gh-140009: Optimize
dict.items()symmetric difference viaPyTuple_FromArray#144771 - gh-140009: Optimize JSON parsing with
object_pairs_hookviaPyTuple_FromArray#144772 - gh-140009: Replace PyTuple_Pack with PyTuple_FromArray in _PyCode_ConstantKey #144773
- gh-140009: Replace PyTuple_Pack with PyTuple_FromArray in codeobject.c and _pickle.c #144829
Pinned by eendebakpt
eendebakptNote: the remaining cases of PyTuple_Pack(1, ...) and PyTuple_Pack(2, ...) are for the most part not performance critical. If opening a PR, please create proper benchmarks (using pyperf, hyperfine or some other benchmarking package) showing the performance gain for the change.
Activity
StanFromIreland commentedon Oct 12, 2025
eendebakpt commentedon Oct 12, 2025
gh-140009: Improve performance of `list_extend_dictitems` by using `P…
kurtmetzger commentedon Oct 15, 2025
If I were to work on this, is it basically changing instances that look like this:
result = PyTuple_Pack(1, basestate);to this?
PyObject *items[1] = {basestate}; result = PyTuple_FromArray(items, 1);for all instances where the array would only be 1 or two items?
Thanks
eendebakpt commentedon Oct 15, 2025
@kurtmetzger Yes, that would be it! But I would wait with pr's for #140132 (comment). If that is accepted, the
PyTuple_FromSinglewould be even better.kurtmetzger commentedon Oct 15, 2025
@eendebakpt I see, so if the other gets accepted, it creates custom functions for arrays of lengths 1 and 2 that should be used like so:
PyTuple_MakePair(a, b)or
PyTuple_MakeSingle(a)to replace the current PyTuple_Pack calls?
eendebakpt commentedon Oct 16, 2025
Yes. Note however that #140079 by @sergey-miryanov is there as well, which overlaps.
Add NEWS entry for pythongh-140009
pythongh-140009: Optimize dict object by replacing PyTuple_Pack with …
pythongh-140009: Optimize dict object by replacing PyTuple_Pack with …
eendebakpt commentedon Feb 8, 2026
Note: the remaining cases of
PyTuple_Pack(1, ...)andPyTuple_Pack(2, ...)are for the most part not performance critical. If opening a PR, please create proper benchmarks (using pyperf, hyperfine or some other benchmarking package) showing the performance gain for the change.pythongh-140009: Replace PyTuple_Pack with PyTuple_FromArray in _warn…
pythongh-140009: Replace PyTuple_Pack with PyTuple_FromArray in _warn…
pythongh-140009: Use PyTuple_FromArray in dictitems_xor_lock_held
pythongh-140009: Use PyTuple_FromArray in _json object_pairs_hook path
pythongh-140009: Replace PyTuple_Pack with PyTuple_FromArray in _PyCo…
pythongh-140009: Use PyTuple_FromArray in dictitems_xor_lock_held
pythongh-140009: Add news entry for json object_pairs_hook optimization
pythongh-140009: Add news entry for dict.items() symmetric difference…
pythongh-140009: Trigger CI rerun
eendebakpt commentedon May 19, 2026
Closing this. The most relevant cases have been covered by #145247