Repository navigation
Some multiprocessing APIs should be possible to import or use even when multiprocessing itself isn't available #134634
Description
Bug report
Bug description:
Android doesn't support multiprocessing because it doesn't support the System V IPC API, and iOS doesn't support subprocesses at all. However, there are a couple of things that should still work on these platforms, but currently don't:
-
multiprocessing.dummy.Pool(akamultiprocessing.pool.ThreadPool) currently depends on themultiprocessingsynchronization primitives, when it could use thethreadingequivalents instead. -
multiprocessing.synchronizethrows an exception on import if the synchronization primitives are unavailable. But this breaks some packages like joblib, which imports the primitives but doesn't always use them (joblib/joblib#825).
Chaquopy has some monkey patches to work around these issues, but it would be better if they were fixed upstream.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Other
Activity
terryjreedy commentedon May 24, 2025
(EDIT) I have no idea is these changed would be acceptible. In any case, we cannot/will-not grab other people's code. Someone with 'ownership' of chaquopy would have to sign our Contributor Licensing Agreement and submit a PR with their code. Or someone else (you?) would have to write new code that does not violate their copyright.
freakboy3742 commentedon May 25, 2025
@terryjreedy FTR: @mhsmith is the founder and principal author of Chaquopy.
mhsmith commentedon May 25, 2025
Yes, I've already signed the CLA, and, if the fact that it's MIT licensed isn't already sufficient, I hereby give permission for anyone to contribute Chaquopy code to CPython.
However, a proper upstream solution will probably look a bit different to the monkey patches. I only linked to them to demonstrate that this could be fixed with a relatively small change.
pythongh-134634: Use a pipe instead of a queue for Pool's change noti…
Merge branch 'main' into pythongh-134634-pool-pipe-notifier
serhiy-storchaka commentedon Apr 4, 2026
I agree that
ThreadPoolshould not depend on shared memory and other IPC. But solution proposed in #148050 while simple, looks heavyweight to me.ThreadPoolshould not need opening named pipes.