Skip to content

Performance of attribute lookup for type objects #92216

Closed
Closed
Performance of attribute lookup for type objects#92216
Labels
3.12only security fixesperformancePerformance or resource usagetype-bugAn unexpected behavior, bug, or error

Description

@eendebakpt
eendebakpt
opened on May 3, 2022

Bug report

The performance of attribute lookup for type objects is worse than for other objects. A benchmark

import pyperf
runner=pyperf.Runner()

setup="""
class Class:
    def all(self):
        pass

x=Class()
"""

runner.timeit('hasattr x.all', "hasattr(x, 'all')", setup=setup)
runner.timeit('hasattr x.__array_ufunc__', "hasattr(x, '__array_ufunc__')", setup=setup)
runner.timeit('hasattr Class.all', "hasattr(Class, 'all')", setup=setup)
runner.timeit('hasattr Class.__array_ufunc__', "hasattr(Class, '__array_ufunc__')", setup=setup) # worse performance

Results:

hasattr x.all: Mean +- std dev: 68.1 ns +- 1.1 ns
hasattr x.__array_ufunc__: Mean +- std dev: 40.4 ns +- 0.3 ns
hasattr Class.all: Mean +- std dev: 38.1 ns +- 0.6 ns
hasattr Class.__array_ufunc__: Mean +- std dev: 255 ns +- 2 ns

The reason seems to be that the type_getattro always executes PyErr_Format, wheras for the "normal" attribute lookup this is avoided (see here and here)

Notes:

Your environment

  • CPython versions tested on: Python 3.11.0a7+
  • Operating system and architecture: Linux Ubuntu

Linked PRs

Activity

  1. eendebakpt
    added
    type-bugAn unexpected behavior, bug, or error
    on May 3, 2022
  2. hugovk
    added
    performancePerformance or resource usage
    on May 3, 2022

  3. seberg commented on May 3, 2022

    @seberg
    seberg
    on May 3, 2022
    Contributor

    Related to this issue, I am wondering if we can consider _PyObject_LookupAttr either as stable (removing the _) or as "semi stable"?
    Of course that is only faster currently for the default tp_getattro, but I suspect that is a huge amount of calls.

    EDIT: Ah, I half thought it was not fully internal API right now, but "semi" exposed.

  4. eendebakpt commented on May 3, 2022

    @eendebakpt
    eendebakpt
    on May 3, 2022
    ContributorAuthor

    Also see: #76752 (PRs where _PyObject_LookupAttr is introduced)

  5. serhiy-storchaka commented on May 4, 2022

    @serhiy-storchaka
    serhiy-storchaka
    on May 4, 2022
    Member

    _PyObject_LookupAttr is an internal API.

    I'll look if it is possible to speed up lookup of the absent attribute in a class object.

  6. seberg
    added a commit that references this issue on May 11, 2022

    PERF: Skip probing `__array_ufunc__` for NumPy builtin scalars (#21470)

    7987557
  7. iritkatriel
    added
    3.12only security fixes
    on Sep 12, 2022
  8. Fidget-Spinner
    added a commit that references this issue on Dec 23, 2022

    gh-92216: improve performance of `hasattr` for type objects (GH-99979)

    7fc7909

  9. Fidget-Spinner commented on Dec 23, 2022

    @Fidget-Spinner
    Fidget-Spinner
    on Dec 23, 2022
    Member

    @eendebakpt thanks for the improvement. Can I close this issue now or do you plan to submit more PRs?

  10. eendebakpt commented on Dec 23, 2022

    @eendebakpt
    eendebakpt
    on Dec 23, 2022
    ContributorAuthor

    The issue is solved, i will close

  11. eendebakpt
    closed this as completedon Dec 23, 2022
  12. iritkatriel
    added a commit that references this issue on Dec 28, 2022

    Merged main into regmachine branch (#50)

    fdc73cb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.12only security fixesperformancePerformance or resource usagetype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions