Repository navigation
raise a Warning when os.fork() is called and the process has multiple threads #100228
Description
Activity
Thanks for adding this!
Reacted by Gregory P. Smith and Mufan QiuWhy a
DeprecationWarning? This suggests that something is deprecated while it is not...ResourceWarningwould sound more appropriate IMHO.Yeah, that question came up on the PR: https://github.com/python/cpython/pull/100229/files#r1048391023
This is one of those cases where we worry about being too noisy if we use a warning that people see outside of their test environment (where deprecation warnings tend to be turned on, they're off by default otherwise).
Lots of code in the world for better or worse "works fine" calling fork() from a threaded application because the software stacks that so many things build on top of don't care: glibc and it's malloc being the prime example. POSIX says it is never okay to do it, but because it worked on old glibc's and glibc malloc: that implementation gets maintained in such a super conservative manner that it does not as a general rule cause problems with threads+fork itself. Other things do, other libc implementations, many other malloc implementations (which often for much better performance specifically in the face of threading), other libraries code may use, etc.
So we want it to be a soft nudge to start with. We can "increase" the verbosity to a different warning type in the future. Or if this proves too disruptive as is, reconsider. Unfortunately we don't have an existing Warning class for this kind of situation.
A perhaps more appropriately named subclass (so that it is hidden by default) of DeprecationWarning could be created so that it at least doesn't sound like we're removing the ability for people to try and shoot themselves in the foot in the future?
I'm expecting we'll hear from users about the noise level by the time we're in the beta phase.
Relevant discussion about this warning in 3.12 release candidate end user testing: https://discuss.python.org/t/concerns-regarding-deprecation-of-fork-with-alive-threads/33555
- added a commit that references this issue
on Sep 24, 2023 - added a commit that references this issue
on Sep 5, 2024
os.fork() in a multi-threaded application is a likely source of deadlocks on many platforms. We should raise a warning when people call os.fork() from a process that we know has other threads running.
This came from discussion in https://discuss.python.org/t/switching-default-multiprocessing-context-to-spawn-on-posix-as-well/21868/, though I believe many of us have pondered doing it in the past.
Linked PRs