Repository navigation
Structural subtype XX of X not recognized as subclass of X in class Z(Generic[TypeVar("XT", bound=X)]) and class ZZ(Z[XX]) #9560
Description
Bug Report
It seems that whereas Python's structural subtyping works (using Python 3.7 and typing_extensions.Protocol) at runtime (see below), mypy doesn't recognize the subtype relationship in the following scenario:
- There is a protocol class
pep249.Connection(I'm creating this). - There is a class
pymysql.connections.Connectionthat conforms to this protocol, as evidenced byissubclass(pymysql.connections.Connection, pep249.Connection) == True. - When defining a generic base class
DatabaseAdapterthat's generic in aConnectionTtype variable that haspep249.Connectionas an upper bound (DatabaseAdapter(Generic[ConnectionT])), and defining a subclassMySQLAdapter
that parameterizes the generic base class withpymysql.connections.Connection(MySQLAdapter(DatabaseAdapter[pymysql.connections.Connection])),mypycomplains thatpymysql.connections.Connectionis not a subtype ofpep249.Connection.
To Reproduce
https://gist.github.com/jmehnle/37ece52f9dbbec439ed8d0c11db47f41
# pep249.py
from typing import Any
from typing_extensions import Protocol, runtime_checkable
@runtime_checkable
class Connection(Protocol):
def close(self) -> None: ...
def commit(self) -> None: ...
def rollback(self) -> None: ...
def cursor(self, *__args: Any, **__kwargs: Any) -> Any: ...# main.py
from typing import Generic, TypeVar
import pep249
import pymysql
ConnectionT = TypeVar("ConnectionT", bound=pep249.Connection)
class DatabaseAdapter(Generic[ConnectionT]): ...
class MySQLAdapter(DatabaseAdapter[pymysql.connections.Connection]): ...Expected Behavior
pymysql.connections.Connection should be recognized as a (structural) subtype of pep249.Connection.
Actual Behavior
(py3.7-typing-protocol) jbook:~/scratch> mypy --version
mypy 0.790+dev.7273e9ab1664b59a74d9bd1d2361bbeb9864b7ab
(py3.7-typing-protocol) jbook:~/scratch> mypy .
main.py:11: error: Type argument "pymysql.connections.Connection" of "DatabaseAdapter" must be a subtype of "pep249.Connection"
Found 1 error in 1 file (checked 2 source files)
(py3.7-typing-protocol) jbook:~/scratch> python
Python 3.7.9 (default, Sep 6 2020, 13:20:25)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pep249
>>> import pymysql
>>> pymysql.connections.Connection
<class 'pymysql.connections.Connection'>
>>> issubclass(pymysql.connections.Connection, pep249.Connection)
True
Your Environment
- Mypy version used: 0.790+dev.7273e9ab1664b59a74d9bd1d2361bbeb9864b7ab (latest master)
- Mypy command-line flags: (none)
- Mypy configuration options from
mypy.ini(and other config files): (none) - Python version used: 3.7.9
- Operating system and version: macOS 10.15.6 with MacPorts
Activity
[-]Structural subtype Y of X not recognized as subclass of X in `class Z(Generic[TypeVar("XT", bound=X)])`[/-][+]Structural subtype XX of X not recognized as subclass of X in `class Z(Generic[TypeVar("XT", bound=X)])` and `class ZZ(Z[X])`[/+][-]Structural subtype XX of X not recognized as subclass of X in `class Z(Generic[TypeVar("XT", bound=X)])` and `class ZZ(Z[X])`[/-][+]Structural subtype XX of X not recognized as subclass of X in `class Z(Generic[TypeVar("XT", bound=X)])` and `class ZZ(Z[XX])`[/+]gvanrossum commentedon Oct 8, 2020
jmehnle commentedon Oct 8, 2020
I'm happy to investigate, but what exactly would I need to look for? Also, what about the fact that
issubclass(in conjunction withtyping_extensions.runtime_checkable) thinks thatpymysql.connections.Connectionis a subclass ofpep249.Connection?hauntsaninja commentedon Oct 8, 2020
issubclass is actually pretty unsophisticated; it only checks presence of an attribute, whereas mypy checks a lot more stuff.
You could take a look at the pymysql stubs: https://github.com/python/typeshed/tree/master/third_party/2and3/pymysql If you can construct a self-contained repro, that would narrow down whether there's an issue with pymysql stubs or a mypy bug. I'd also confirm whether something simpler like:
x: pep249.Connection = pymysql.connections.Connection(...)passes.There are a couple other open issues for mypy to be more helpful about diagnosing why something isn't an instance of a Protocol — I think improving this is a pretty high priority.
jmehnle commentedon Oct 9, 2020
@hauntsaninja, the advice to try a plain assignment was gold. It gets me some useful output:
So if I change the method definition like so:
then it all works. But this puts me in a dilemma, because PEP 249 doesn't specify the signature of
Connection.cursor(), and I can't assume there's a single argument (with a default), because other, non-pymysqlDBI drivers have different arguments. Based on #5876 there's no way to type an arbitrary argument list. Evidently my first attempt doesn't match the actual signature ofpymysql.connections.Connection.cursor.gvanrossum commentedon Oct 9, 2020
Here's a horrible hack that might help: declare
cursoras a property (really!) that returns a Callable:No errors there.
jmehnle commentedon Oct 12, 2020
Hot damn, that works. Thanks, @gvanrossum. I never, ever would've come up with that trick. So where does this leave us? I see at least a couple deficiencies:
Type argument "XX" of "ZZ" must be a subtype of "X"is triggered,mypydoesn't give the same helpful information about whyXXisn't recognized as a subtype ofXas it does whenIncompatible types in assignmentis triggered.selfrequired, all further arguments unspecified) in a protocol class that structural subtypes can satisfy (Make any callable compatible with (*args: Any, **kwargs: Any)? #5876). Thepropertytrick basically ignores all arguments, which happens to work well enough in this case.hauntsaninja commentedon Feb 1, 2023
Closing, since #5876 has been fixed