Crash report
What happened?
While working on gh-126644, I came across an elusive bug that seems to occur when a lot of threads are trying to create a thread state for the same interpreter. Initially, I thought it was an issue with free-threading, but it occurs on the default build as well.
Here's a small reproducer:
from threading import Thread
import _interpreters
interp = _interpreters.create()
def run():
this_interp = _interpreters.create()
_interpreters.run_string(this_interp, f"import _interpreters; _interpreters.run_string({interp}, '1')")
threads = [Thread(target=run) for _ in range(1000)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()As it turns out, this is because of this check: https://github.com/python/cpython/blob/main/Python/pystate.c#L1531
Inside tstate_delete_common, the runtime lock is released after the threads.head has already been set to NULL, so other threads think erroneously think it's OK to try and use _initial_thread while it's still getting finalized. Possibly related, free_threadstate tries to reset _initial_thread back to the default settings without the runtime lock, which is possibly racy.
There's a few ways to fix this, but I think the easiest (and has the least amount of risk for backporting) is to just hold the runtime lock for all of thread state deletion. This will introduce a very slight slowdown due to lock contention, but that should get better once #114940 lands. I have a PR ready.
CPython versions tested on:
3.12, 3.13, 3.14, CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
No response
Linked PRs
Crash report
What happened?
While working on gh-126644, I came across an elusive bug that seems to occur when a lot of threads are trying to create a thread state for the same interpreter. Initially, I thought it was an issue with free-threading, but it occurs on the default build as well.
Here's a small reproducer:
As it turns out, this is because of this check: https://github.com/python/cpython/blob/main/Python/pystate.c#L1531
Inside
tstate_delete_common, the runtime lock is released after thethreads.headhas already been set toNULL, so other threads think erroneously think it's OK to try and use_initial_threadwhile it's still getting finalized. Possibly related,free_threadstatetries to reset_initial_threadback to the default settings without the runtime lock, which is possibly racy.There's a few ways to fix this, but I think the easiest (and has the least amount of risk for backporting) is to just hold the runtime lock for all of thread state deletion. This will introduce a very slight slowdown due to lock contention, but that should get better once #114940 lands. I have a PR ready.
CPython versions tested on:
3.12, 3.13, 3.14, CPython main branch
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
No response
Linked PRs