Repository navigation
multiprocessing classes SimpleQueue and Queue don't support typing in 3.11.0 #99509
Description
Activity
- added a commit that references this issue
on Nov 15, 2022 - added a commit that references this issue
on Dec 27, 2022 - added a commit that references this issue
on Dec 28, 2022 I am using Python 3.11.4 and still get this error:
from multiprocessing import Queue q: Queue[str] = Queue()
Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'method' object is not subscriptableAnd using
import multiprocessing from multiprocessing.queues import Queue q: Queue[str] = multiprocessing.Queue()
Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: type 'Queue' is not subscriptableSimpleQueue subscripting only works when imported using
from multiprocessing.queues import SimpleQueue.I believe this should be re-opened.
As reviewers noted in the linked PR, the linked PR only solves it for an undocumented way to import Queue. Also, even with the new import it's not a drop in replacement. I tested with 3.12beta 3.
from multiprocessing.queues import Queue queue: Queue[str] = Queue() print(queue)
The new import yields an error.
Traceback (most recent call last): File "myfilelocation\test.py", line 3, in <module> queue: Queue[str] = Queue() ^^^^^^^ TypeError: Queue.__init__() missing 1 required keyword-only argument: 'ctx'To work around subscripting after this PR, you need to use the undocumented queue import and then provide this unknown context argument directly.
from multiprocessing.queues import Queue import multiprocessing queue: Queue[str] = Queue(ctx=multiprocessing.get_context()) print(queue)
And of course the documented way of making a
Queuestill fails.from multiprocessing import Queue queue: Queue[str] = Queue() print(queue)
Traceback (most recent call last): File "myfilelocation\test2.py", line 3, in <module> queue: Queue[str] = Queue() ~~~~~^^^^^ TypeError: 'method' object is not subscriptableReacted by Matteo Golin, hongwen000 and Simon H@Starbuck5 Same issue
To make both mypy and CPython happy, you can do something like this:
import multiprocessing import multiprocessing.queues queue: multiprocessing.queues.Queue[str] = multiprocessing.Queue()
This accurately reflects the facts that:
multiprocessing.Queue()is the documented, public-API way of creating the class; however,multiprocessing.Queueitself is not a type (it's a method), so can't be used in a type annotation
Could we special-case
multiprocessing.Queue, perhaps by pretending in typeshed that it's a class rather than a method? Perhaps we could, but we like to avoid doing that kind of thing as much as possible. It's generally best for type checkers to be told the truth; lying to type checkers in typeshed about whether things are methods or classes can have unfortunate consequences. In any event, that conversation would probably need to take place on the https://github.com/python/typeshed/issues issue tracker rather than at CPython.Could
multiprocessing.Queueitself be changed so that it's an actual class rather than a method? Again, perhaps. I'm not a multiprocessing expert, though (just a typing expert), so I can't really comment on that one.Reacted by YANG Qi, AndrewCleveland-Hearst, Shir Amir, Anton Yablokov, Jianan Wang and rysson
Bug report
SimpleQueueandQueueclasses frommultiprocessingmodule in Python 3.11.0 do not support type[str]annotation.Minimal, reproducible example:
or
Result - error:
How it should work:
It should work like
Queuefrom thequeuemodule:Result - no error.
Why do I need this?
I want my IDE to know that
queue.get()returnsstrobject.Your environment
Python 3.11.0 arm64
Python 3.11.0 (main, Nov 4 2022, 17:22:54) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
MacBook M1 Pro macOS Ventura 13.0.1.
Linked PRs
__class_getitem__tomultiprocessing.Queue#99511