singledispatchmethod raises an error when relying on a forward declaration

@gvanrossum

BPO	41987
Nosy	@gvanrossum, @glyph, @ambv, @ethanhs, @joernheissler, @isidentical, @mental32, @ryansobol, @AlexWaygood
PRs	bpo-41987: Fix unnecessary evaluation of return type forward declaratons in singledispatch. #23216

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-10-09.20:11:43.528>
labels = ['type-bug', 'library', '3.9', '3.10']
title = 'singledispatchmethod raises an error when relying on a forward declaration'
updated_at = <Date 2022-01-11.09:46:19.220>
user = 'https://github.com/glyph'

bugs.python.org fields:

activity = <Date 2022-01-11.09:46:19.220>
actor = 'AlexWaygood'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-10-09.20:11:43.528>
creator = 'glyph'
dependencies = []
files = []
hgrepos = []
issue_num = 41987
keywords = ['patch']
message_count = 16.0
messages = ['378346', '378350', '378353', '378361', '379896', '379911', '380797', '380799', '380800', '380803', '380832', '380843', '380845', '380852', '409572', '410269']
nosy_count = 9.0
nosy_names = ['gvanrossum', 'glyph', 'lukasz.langa', 'ethan smith', 'joernheissler', 'BTaskaya', 'mental', 'ryansobol', 'AlexWaygood']
pr_nums = ['23216']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41987'
versions = ['Python 3.9', 'Python 3.10']

This example:

from __future__ import annotations
from functools import singledispatchmethod


class Comparable:
    @singledispatchmethod
    def compare(self, arg: object):
        raise NotImplementedError("what")

    @compare.register
    def _(self, arg: Comparable):
        return "somewhat similar"

print(Comparable().compare(Comparable()))

Produces this result:

File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/typing.py", line 518, in _evaluate
eval(self.__forward_code__, globalns, localns),
File "<string>", line 1, in <module>
NameError: name 'Comparable' is not defined

It seems like perhaps singledispatchmethod should defer its type evaluation to its first invocation?

AFAIK the normal way of registering types (dispatcher.register(<type>)) also requires that registered type be defined at the execution type. I guess, if we are going to support such a thing, we might end up with supporting passing strings into the .register() as the initial argument of matching type to be consistent. Anyways, this would be a feature request for 3.10+, so changing the version info.

This behavior (only relevant with from __future__ import annotations) has been around since @singledispatchmethod was introduced in 3.8, so I agree we should treat it as a feature request.

In the meantime, maybe a workaround is to move the register call out of the class? It looks a little bit ugly but probably works.

(Disclaimer: I'm not familiar with singledispatchmethod.)

The behavior is the same with a traditional quoted forward declaration, so it’s not specific to the __future__ import; I just phrased the example that way to show how it’s going to look in the future and to illustrate how it might crop up in a way which is maximally confusing to users less familiar with the internals of type annotations.

It's worth pointing out that a similar error is produced for a forward-referenced return type of a registered method, but only for python3.9. For example:

from __future__ import annotations
from functools import singledispatchmethod


class Integer:
    def __init__(self, value: int):
        self.value = value

    def __str__(self) -> str:
        return str(self.value)

    @singledispatchmethod
    def add(self, other: object) -> Integer:
        raise NotImplementedError(f"Unsupported type {type(other)}")

    @add.register
    def _(self, other: int) -> "Integer":
        return Integer(self.value + other)

print(Integer(2).add(40))

This code runs without error in python3.8, and I am using this technique in code running in a production environment.

$ python3.8 --version
Python 3.8.6
$ python3.8 integer.py
42

However, this code throws a NameError in python3.9.

$ python3.9 --version
Python 3.9.0
$ python3.9 integer.py
Traceback (most recent call last):
  File "/Users/ryansobol/Downloads/integer.py", line 5, in <module>
    class Integer:
  File "/Users/ryansobol/Downloads/integer.py", line 17, in Integer
    def _(self, other: int) -> "Integer":
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/functools.py", line 909, in register
    return self.dispatcher.register(cls, func=method)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/functools.py", line 860, in register
    argname, cls = next(iter(get_type_hints(func).items()))
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 1386, in get_type_hints
    value = _eval_type(value, globalns, localns)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 254, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 497, in _evaluate
    self.__forward_value__ = _eval_type(
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 254, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "/usr/local/Cellar/python@3.9/3.9.0_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 493, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'Integer' is not defined

I know that some may see this issue as a feature request for 3.10+. However, for me, it is a bug preventing my code from migrating to 3.9.

Hello!

git-bisect points at
https://bugs.python.org/issue41341
#21553

It breaks both the examples from
https://bugs.python.org/issue41987#msg379896 and
https://bugs.python.org/issue41987#msg380803

Thanks for the bisection. It's not surprising that that's the culprit, and in other situations that's the right thing to do. I'm not sure how to address this without breaking other stuff -- maybe leave the ForwardRef if evaluating it doesn't work? But that's likely to have other subtle side effects -- we still want simple typos (or other reasons why a reference is legitimately broken) to go unchecked. Maybe singledispatch can catch the error and fall back on looking at bare __annotations__?

pythonGH-86153: get_type_hints now parses forward reference properly

pythonGH-86153: Created a test for get_type_hints based on GVR's exam…

This can probably be closed as it's not an issue in Py 3.10+

bintime: Work around python/cpython#86153

Add bintime submodule (#73)

@isidentical

This can probably be closed as it's not an issue in Py 3.10+

I can still reproduce the issue, so I can't confirm this. What exactly changed in Python 3.10?

First, regarding the problem discussed in #86153 (comment) and further on: this remained unfixed in 3.9, 3.10, 3.11, 3.12 and 3.13. It was fixed by PEP 749, precisely by 7b7b90d (GH-119891):

t.py:

from __future__ import annotations
from functools import singledispatchmethod

class Integer:
    @singledispatchmethod
    def add(self, other: object) -> Integer:
        raise NotImplementedError(f"Unsupported type {type(other)}")

    @add.register
    def _(self, other: int) -> "Integer":
        ...

print("ok!")

❯ uvx every-python run 7b7b90d~ python t.py |& tail -n 1
NameError: name 'Integer' is not defined

❯ uvx every-python run 7b7b90d python t.py |& tail -n 1
ok!

Therefore, this is no longer a bug to fix.

The original error reported by the OP still reproduces on 3.15:

from __future__ import annotations
from functools import singledispatchmethod

class Comparable:
    @singledispatchmethod
    def compare(self, arg: object):
        raise NotImplementedError("what")

    @compare.register
    def _(self, arg: Comparable):
        return "somewhat similar"

with a lovely error message:

Traceback (most recent call last):
  File "/home/bswck/Python/cpython/t.py", line 33, in <module>
    class Comparable:
    ...<6 lines>...
            return "somewhat similar"
  File "/home/bswck/Python/cpython/t.py", line 38, in Comparable
    @compare.register
     ^^^^^^^^^^^^^^^^
  File "/home/bswck/Python/cpython/Lib/functools.py", line 1030, in register
    return self.dispatcher.register(cls, func=method)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bswck/Python/cpython/Lib/functools.py", line 974, in register
    raise TypeError(
    ...<2 lines>...
    )
TypeError: Invalid annotation for 'arg'. ForwardRef('Comparable') is an unresolved forward reference.

(Without from __future__ import annotations on top we also get nearly the same error).

It's not a bug report, but a feature request, as previously noted by @isidentical in #86153 (comment) and by @gvanrossum in #86153 (comment). I'm relabeling this issue accordingly.

As @isidentical noted, the reason why this didn't work in the first place is that @register evaluates the annotation immediately, and at the time of evaluation the class body is still executing to complete the class namespace; the final class object is not available in any way yet. There is no way to get to the class object, because it simply doesn't exist at this time.

One simple solution that comes to my mind is delaying registrations entirely by moving them to __set_name__ of singledispatchmethod, when the class object is ready.
singledispatchmethod.register() would simply queue registrations and __set_name__ would flush it to the singledispatch registry in the context of the finalized class.

Doing this to all registrations could break .dispatch calls inside the class body, but that seems like poor programming to me.
If that is a concern though, we could only queue unresolvable registrations, but that gives us a complicated flow to reason about + implementation complexity.

Revisiting this, I think it's not worth it.

This is easy and works:

from __future__ import annotations
from functools import singledispatchmethod

class Comparable:
    @singledispatchmethod
    def compare(self, arg: object):
        raise NotImplementedError("what")

    def callback(self, arg: Comparable):
        return "somewhat similar"
    
Comparable.compare.register(Comparable.callback)

The cost of changing the assumptions that singledispatch was built on is IMO too high to support this one edge case.

@johnslavik

@johnslavik the suggested workaround:

from __future__ import annotations
from functools import singledispatchmethod

class Comparable:
    @singledispatchmethod
    def compare(self, arg: object):
        raise NotImplementedError("what")

    def callback(self, arg: Comparable):
        return "somewhat similar"
    
Comparable.compare.register(Comparable.callback)

does work at runtime, but results in a mypy error: error: "Callable[..., Any]" has no attribute "register" [attr-defined]

Navigation Menu

Repository navigation

singledispatchmethod raises an error when relying on a forward declaration #86153

11 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

singledispatchmethod raises an error when relying on a forward declaration #86153

Description

Activity

glyph commented on Oct 9, 2020

isidentical commented on Oct 9, 2020

gvanrossum commented on Oct 9, 2020

glyph commented on Oct 10, 2020

ryansobol commented on Oct 29, 2020

11 remaining items

joernheissler commented on Jan 3, 2022

gvanrossum commented on Jan 11, 2022

ashb commented on Jan 13, 2025

johnslavik commented on Jan 6, 2026

johnslavik commented on Jan 20, 2026

kyleaoman commented on May 20, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions