Skip to content

Deadlock when two threads first resolve PyDecimalType at the same time #153

Closed

Description

@quantconnect-kronos

Expected Behavior

Two threads that each take the GIL and call into .NET from Python at the same time, for the first time in the process, both complete.

Actual Behavior

They can deadlock on Runtime.PyDecimalType.

PyDecimalType is created in Runtime.InitPyMembers as a Lazy<PyType> with the default LazyThreadSafetyMode.ExecutionAndPublication. Its factory imports _pydecimal, which is a pure Python module, while the calling thread holds the GIL. While that module's bytecode runs, the interpreter can hand the GIL to another thread. If that thread then reaches Converter.GetTypeByAlias (from MethodBinding.tp_call, MethodBinder or Runtime.PythonArgsToTypeArray), it blocks on the Lazy's lock while it still holds the GIL. The first thread can't get the GIL back to finish the import, so neither thread moves on.

Stacks from a process stuck this way (trimmed):

Thread A (holds the GIL, waits on the Lazy's lock)
  System.Threading.Monitor.Enter_Slowpath
  System.Lazy`1.ExecutionAndPublication
  System.Lazy`1.CreateValue
  Python.Runtime.Converter.GetTypeByAlias
  Python.Runtime.MethodBinding.tp_call
  Python.Runtime.Runtime.PyObject_Call
  Python.Runtime.PyObject.Invoke

Thread B (holds the Lazy's lock, waits for the GIL inside the import)
  [Native Frames]
  Python.Runtime.Runtime.PyImport_ImportModule
  Python.Runtime.Runtime+<>c.<InitPyMembers>b__31_0
  System.Lazy`1.ViaFactory
  System.Lazy`1.ExecutionAndPublication
  System.Lazy`1.CreateValue
  Python.Runtime.Converter.GetTypeByAlias
  Python.Runtime.MethodBinding.tp_call
  Python.Runtime.Runtime.PyObject_Call

It only happens while the value hasn't been created yet, so it shows up when several threads start running Python at the same moment, such as parallel NUnit tests.

Potential Solution

Don't hold a lock while the factory runs Python code. For example, create the Lazy with LazyThreadSafetyMode.PublicationOnly: two threads may both import _pydecimal, which Python's import system handles, and the first result wins. Creating the type eagerly in InitPyMembers would also work, at the cost of importing _pydecimal at startup.

Reproducing the Problem

Run several threads that each take the GIL with Py.GIL() and call an overloaded .NET method from Python as their first call, with a non-primitive argument so that GetTypeByAlias reaches the PyDecimalType check. The hang comes and goes because it depends on when the interpreter switches threads during the _pydecimal import. Creating PyDecimalType's value once on a single thread before the threads start makes it go away.

System Information

QuantConnect.pythonnet 2.0.67, .NET 10, Linux.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions