singledispatchmethod significantly slower than singledispatch #85160

Activity

1. CaselIT commented on Jun 15, 2020

   

   CaselITmannequin

   on Jun 15, 2020

   MannequinAuthor

   More actions

   The implementation of singledispatchmethod is significantly slower (~4x) than the normal singledispatch version

   Using timeit to test this example case:

       from functools import singledispatch, singledispatchmethod
       import timeit
   
       class Test:
           @singledispatchmethod
           def go(self, item, arg):
               print('general')
           
           @go.register
           def _(self, item:int, arg):
               return item + arg
   
       @singledispatch
       def go(item, arg):
           print('general')
   
       @go.register
       def _(item:int, arg):
           return item + arg

   print(timeit.timeit('t.go(1, 1)', globals={'t': Test()}))
   print(timeit.timeit('go(1, 1)', globals={'go': go}))
   

   Prints on my system.

   3.118346
   0.713173
   

   Looking at the singledispatchmethod implementation I believe that most of the difference is because a new function is generated every time the method is called.

   Maybe an implementation similar to cached_property could be used if the class has __dict__ attribute?
   Trying this simple patch

   diff --git a/Lib/functools.py b/Lib/functools.py
   index 5cab497..e42f485 100644
   --- a/Lib/functools.py
   +++ b/Lib/functools.py
   @@ -900,6 +900,7 @@ class singledispatchmethod:
   

               self.dispatcher = singledispatch(func)
               self.func = func
       +        self.attrname = None
   
           def register(self, cls, method=None):
               """generic_method.register(cls, func) -> func
       @@ -908,6 +909,10 @@ class singledispatchmethod:
               """
               return self.dispatcher.register(cls, func=method)

   +    def __set_name__(self, owner, name):
   +        if self.attrname is None:
   +            self.attrname = name
   +
       def __get__(self, obj, cls=None):
           def _method(*args, **kwargs):
               method = self.dispatcher.dispatch(args[0].__class__)
   @@ -916,6 +921,7 @@ class singledispatchmethod:
           _method.__isabstractmethod__ = self.__isabstractmethod__
           _method.register = self.register
           update_wrapper(_method, self.func)
   +        obj.__dict__[self.attrname] = _method
           return _method
   
       @property
   

   improves the performance noticeably

   0.9720976
   0.7269078
   
2. CaselIT

   added

   3.8 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   performancePerformance or resource usagePerformance or resource usage

   on Jun 15, 2020
3. ezio-melotti

   transferred this issue fromon Apr 10, 2022
4. iritkatriel

   added

   3.12only security fixesonly security fixes

   and removed

   3.8 (EOL)end of lifeend of life

   on Sep 9, 2022
5. iritkatriel commented on Sep 9, 2022

   

   iritkatriel

   on Sep 9, 2022

   Member

   More actions

   I see a similar time difference on main.
6. AlexWaygood

   added

   type-featureA feature request or enhancementA feature request or enhancement

   and removed

   3.12only security fixesonly security fixes

   on Jul 5, 2023
7. AlexWaygood

   added a commit that references this issue on Aug 6, 2023

   gh-85160: improve performance of `functools.singledispatchmethod` (#1…

   ...

   3e334ae
8. AlexWaygood commented on Aug 6, 2023

   

   AlexWaygood

   on Aug 6, 2023

   Member

   More actions

   #107148 significantly optimised @singledispatchmethod for all non-slotted classes. If anybody can think of further optimisations that can be safely applied, I suggest a new issue should be opened for them.

   Thanks @casselt for the report, and thanks @mental32 and @eendebakpt for working on this!

   ❤️React with ❤️1Reacted by Chris
9. AlexWaygood

   closed this as completedon Aug 6, 2023
10. AlexWaygood

    added 2 commits that reference this issue on Aug 7, 2023

    pythongh-85160: Reduce memory usage of `singledispatchmethod` when in…

    ...

    bfce05c

    gh-85160: Reduce memory usage of `singledispatchmethod` (#107706)

    ...

    2ac103c
11. serhiy-storchaka

    added 2 commits that reference this issue on Feb 11, 2025

    pythongh-127750: Fix and optimize functools.singledispatchmethod()

    ...

    7c5bb0e

    gh-127750: Fix and optimize functools.singledispatchmethod() (GH-130008)

    ...

    395335d