Repository navigation
DictWriter is not subscriptable #7787
Description
mypy complains that DictWriter is missing the type parameter in the code:
from csv import DictWriter
from typing import IO
def f_is_for_fun(file: IO[str]) -> DictWriter:
return DictWriter(file, ["id", "name"])
which is reasonable, because DictWriter is defined as class DictWriter(Generic[_T]) (https://github.com/python/typeshed/blob/master/stdlib/csv.pyi)
But if I change the code to make mypy happy:
from csv import DictWriter
from typing import IO
def f_is_for_fun(file: IO[str]) -> DictWriter[str]:
return DictWriter(file, ["id", "name"])
then I become an error at runtime:
…
def f_is_for_fun(file: IO[str]) -> DictWriter[str]:
TypeError: 'type' object is not subscriptable
or, with python 3.11
TypeError: type 'DictWriter' is not subscriptable
And if I add
from __future__ import annotations
then mypy and the python interpreter are happy, but not pylint.
Activity
clo-vis commentedon May 6, 2022
#4800
AlexWaygood commentedon May 6, 2022
This is a very commonly encountered problem -- there's some useful mypy docs on tackling it here: https://mypy.readthedocs.io/en/stable/runtime_troubles.html#using-classes-that-are-generic-in-stubs-but-not-at-runtime.
For this specific situation, you can get round it by just using quoted annotations, e.g.
CPython is also pretty receptive to feature requests adding
__class_getitem__to classes which are generic in typeshed (e.g. python/cpython#92129).AlexWaygood commentedon May 6, 2022
Wait, why does pylint complain about
__future__annotations??clo-vis commentedon May 6, 2022
pylint has no problem with
__future__, it just complains that DictWriter is not subscriptable (with or without__future__)AlexWaygood commentedon May 6, 2022
That seems like a problem with pylint, do you know if there any bug reports tracking that issue?
clo-vis commentedon May 6, 2022
Coincidentally I know there is a bug report a few minutes old :)
pylint-dev/pylint#6523
AlexWaygood commentedon May 6, 2022
Great. I'm going to close this for now, as I don't think there's anything actionable from the typeshed end on this one. There are some pretty easy workarounds for this specific situation (either use explicit quotes, or use
from __future__ import annotations); the pylint bug is a pylint bug; and if we want to add__class_getitem__to the runtime, that should be discussed over at the CPython issue tracker rather than here.I don't know that I'm 100% confident that the stubs for
csv.DictReaderandcsv.DictWriterare perfect right now, but I'll need to some experimenting before I have fully formed thoughts on that. And it's not really the issue under discussion here, anyway.cdce8p commentedon May 6, 2022
After reading the pylint issue, I opened an issue in cpython python/cpython#92391 to add
__class_getitem__. Happy to provide a PR, not sure though if that's the way to go now though after your last comment @AlexWaygood.I'll have a look at the other pylint issue with
from __future__ import annotationsthough. That seems like a bug.AlexWaygood commentedon May 6, 2022
DictReaderseems pretty clearly like it should be generic -- the type that's returned fromDictReader.__next__depends very clearly on the type that you stick intoDictReader.__init__.In fact,
DictReaderseems like it could possibly be typed more correctly if it took another parameter: see #5994 (comment). (But that's irrelevant to the runtime; adding__class_getitem__will allow it to take any number of parameters at runtime.)I was confused about the implicit conversion to string in
DictWriter, which made me wonder ifDictWritershould be generic. ButDictWriterdoes some runtime typechecking in the_dict_to_listmethod before it does the implicit conversion to string. So I think it makes sense forDictWriterto be generic as well (as it is in the stub currently).My nervousness here is that these haven't been generic in typeshed for that long, and they're not "simple" classes to type. So the stubs might still change in the future.
But I think they'll probably stay generic. So I think this is okay.
Akuli commentedon May 6, 2022
I made
DictWriterwriting enums toreading enums from csv files easily.AlexWaygood commentedon May 6, 2022
I think that was a good change to make. Just took me a little bit of time to get my head around some bits of the stub :)
Correct the not subscriptable error