You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Deadlock when two threads first resolve PyDecimalType at the same time #153
Two threads that each take the GIL and call into .NET from Python at the same time, for the first time in the process, both complete.
Actual Behavior
They can deadlock on Runtime.PyDecimalType.
PyDecimalType is created in Runtime.InitPyMembers as a Lazy<PyType> with the default LazyThreadSafetyMode.ExecutionAndPublication. Its factory imports _pydecimal, which is a pure Python module, while the calling thread holds the GIL. While that module's bytecode runs, the interpreter can hand the GIL to another thread. If that thread then reaches Converter.GetTypeByAlias (from MethodBinding.tp_call, MethodBinder or Runtime.PythonArgsToTypeArray), it blocks on the Lazy's lock while it still holds the GIL. The first thread can't get the GIL back to finish the import, so neither thread moves on.
Stacks from a process stuck this way (trimmed):
Thread A (holds the GIL, waits on the Lazy's lock)
System.Threading.Monitor.Enter_Slowpath
System.Lazy`1.ExecutionAndPublication
System.Lazy`1.CreateValue
Python.Runtime.Converter.GetTypeByAlias
Python.Runtime.MethodBinding.tp_call
Python.Runtime.Runtime.PyObject_Call
Python.Runtime.PyObject.Invoke
Thread B (holds the Lazy's lock, waits for the GIL inside the import)
[Native Frames]
Python.Runtime.Runtime.PyImport_ImportModule
Python.Runtime.Runtime+<>c.<InitPyMembers>b__31_0
System.Lazy`1.ViaFactory
System.Lazy`1.ExecutionAndPublication
System.Lazy`1.CreateValue
Python.Runtime.Converter.GetTypeByAlias
Python.Runtime.MethodBinding.tp_call
Python.Runtime.Runtime.PyObject_Call
It only happens while the value hasn't been created yet, so it shows up when several threads start running Python at the same moment, such as parallel NUnit tests.
Potential Solution
Don't hold a lock while the factory runs Python code. For example, create the Lazy with LazyThreadSafetyMode.PublicationOnly: two threads may both import _pydecimal, which Python's import system handles, and the first result wins. Creating the type eagerly in InitPyMembers would also work, at the cost of importing _pydecimal at startup.
Reproducing the Problem
Run several threads that each take the GIL with Py.GIL() and call an overloaded .NET method from Python as their first call, with a non-primitive argument so that GetTypeByAlias reaches the PyDecimalType check. The hang comes and goes because it depends on when the interpreter switches threads during the _pydecimal import. Creating PyDecimalType's value once on a single thread before the threads start makes it go away.
Activity