Repository navigation
MainThread association logic is fragile #75698
Description
Activity
The threading main_thread() instance is associated to the thread that first imports the threading module. In usual circumstances, this will indeed be the interpreter's main thread. However, it is not always the case. See attached reproducer.
$ ./python mainthread2.py child thread: <_MainThread(MainThread, started 140399567398656)> main thread: <_DummyThread(Dummy-1, started daemon 140399588386560)> Exception ignored in: <module 'threading' from '/home/antoine/cpython/default/Lib/threading.py'> Traceback (most recent call last): File "/home/antoine/cpython/default/Lib/threading.py", line 1268, in _shutdown assert tlock.locked() AssertionError:
Fixing this will require thinking out what the "main thread" of an interpreter really is. Is it the thread in which PyInterpreterState_New() is called?
(note this is not the same thing as the "main thread" in the signal module, which is process-global)
Is there a problem here? I haven't heard of anyone even wondering about this before. threading.py worries about Threads created by threading.py, plus the magical (from its point of view) thread that first imports threading.py. Users mix in
_threadthreads, or raw C threads from extension modules, essentially at their own risk. Which risks are minimal, but can have visible consequences.I don't view that as being a real problem. It might help if, e.g., a Wiki somewhere spelled out the consequences under different Python implementations (while I don't know for sure, I _expect_ the current docs just say "in normal conditions, the main thread is the thread from which the Python interpreter was started" because it doesn't want to over-specify the behavior in an area nobody really cares about).
One place where this came up recently is in working out precisely how a Python-level subinterpreter API will interact with the threading API: https://mail.python.org/pipermail/python-dev/2017-September/149566.html
That said, I do agree with Tim that the status quo isn't broken per se: we renamed
threadto_threadin Python 3 for a reason, and that reason is that you really need to know how Python's threading internals work to do it safely.However, I do think we can treat this as a documentation enhancement request, where the
_threadmodule docs could point out some of the requirements to ensure that low-level thread manipulation plays nice with the threading module.That said, I do agree with Tim that the status quo isn't broken per se: we renamed
threadto_threadin Python 3 for a reasonThis is not caused by
_threadspecifically, but any background thread created by C code that might invoke Python code. The reproducer just uses the_threadmodule out of convenience.I'm not sure this is an issue worth fixing, but I don't think it's a good idea to document such quirky semantics.
We had the quirks of import related threading deadlocks documented for a long time, not as a promise, but as a debugging aid (and a recommendation for "don't do that").
I'd see this as being similar: we'd document that if you're using the _thread module, or otherwise allowing operating system threads not managed by the threading module to call in and run arbitrary Python code, then you should run "import threading" early in the actual main thread to make sure it gets associated correctly.
Python internals already know who is the "main" thread: _PyRuntime.main_thread. It's maintained up to date, even after a fork, PyOS_AfterFork_Child() calls _PyRuntimeState_ReInitThreads() which does:
// This was initially set in _PyRuntimeState_Init(). runtime->main_thread = PyThread_get_thread_ident();I already added _thread._is_main_interpreter() to deny spawning daemon threads in subinterpreters: bpo-37266.
We can add _thread._is_main_thread() which can reuse Modules/signalmodule.c code:
static int is_main(_PyRuntimeState *runtime) { unsigned long thread = PyThread_get_thread_ident(); PyInterpreterState *interp = _PyRuntimeState_GetThreadState(runtime)->interp; return (thread == runtime->main_thread && interp == runtime->interpreters.main); }
For example, this function is used by signal.signal:
if (!is_main(runtime)) { PyErr_SetString(PyExc_ValueError, "signal only works in main thread"); return NULL; }
probably a duplicate: issue bpo-39042 "Use the runtime's main thread ID in the threading module."
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: