Skip to content

Deadlock when mixing event loops and subprocesses #88056

Closed
Closed
Deadlock when mixing event loops and subprocesses#88056
Labels
3.10 (EOL)end of life3.11only security fixes3.9 (EOL)end of lifetopic-asynciotype-bugAn unexpected behavior, bug, or error

Description

@ehrmann
ehrmann
mannequin
opened on Apr 19, 2021
BPO 43890
Nosy @ehrmann
Files
  • subprocess_asyncio_deadlock.py
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2021-04-19.18:36:19.433>
labels = ['type-bug', '3.8']
title = 'Deadlock when mixing event loops and subprocesses'
updated_at = <Date 2021-04-19.18:36:19.433>
user = 'https://github.com/ehrmann'

    bugs.python.org fields:

    activity = <Date 2021-04-19.18:36:19.433>
actor = 'thomas'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2021-04-19.18:36:19.433>
creator = 'thomas'
dependencies = []
files = ['49967']
hgrepos = []
issue_num = 43890
keywords = []
message_count = 1.0
messages = ['391386']
nosy_count = 1.0
nosy_names = ['thomas']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43890'
versions = ['Python 3.8']

    Activity

    1. ehrmann commented on Apr 19, 2021

      @ehrmann
      ehrmannmannequin
      on Apr 19, 2021
      MannequinAuthor

      When mixing code that spawns subprocesses with code that creates event loops, Python appears to deadlock.

      In the attached example, when WORKERS = 16 and ASYNC_WORKERS = 8, Python will sometimes (50% of the time?) deadlock, never exiting, with no exceptions raised. Oddly, if ASYNC_WORKERS is set to 0 or 16 (i.e., only subprocesses or only event loops), it runs reliably.

      I tested this in an x86_64 Ubuntu 20.04.2 VM with Python 3.8.6. I was only able to reproduce it when the VM had more than one CPU, and I was unable to reproduce it on x86_64 MacOS.

    2. ehrmann
      added
      3.8 (EOL)end of life
      type-bugAn unexpected behavior, bug, or error
      on Apr 19, 2021
    3. ezio-melotti
      transferred this issue fromon Apr 10, 2022
    4. kumaraditya303
      added
      3.11only security fixes
      3.10 (EOL)end of life
      3.9 (EOL)end of life
      and removed
      3.8 (EOL)end of life
      on Apr 16, 2022

    5. kumaraditya303 commented on Apr 16, 2022

      @kumaraditya303
      kumaraditya303
      on Apr 16, 2022
      Contributor

      Reproduced on 3.11, I think it is because of fork

    6. AlexWaygood
      added
      topic-asyncio
      on Apr 16, 2022

    7. kumaraditya303 commented on Apr 16, 2022

      @kumaraditya303
      kumaraditya303
      on Apr 16, 2022
      Contributor

      Duplicate of #90368

      So after some investigation, this issue is a duplicate of #90368 and if you remove the print calls it is fixed. Moreover the fix in #30310 fixes this issue so closing as duplicate.

    8. kumaraditya303
      closed this as completedon Apr 16, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of life3.11only security fixes3.9 (EOL)end of lifetopic-asynciotype-bugAn unexpected behavior, bug, or error

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions