IsolatedAsyncioTestCase does not call asyncio.set_event_loop before setUp anymore, asyncio.Runner+PidFdChildWatcher leaves zombie processes

demo reproducer:

import unittest
import asyncio

class DemoTestCase1(unittest.IsolatedAsyncioTestCase):
    def setUp(self):
        self.loop = asyncio.get_event_loop_policy().get_event_loop()

    async def test_demo(self):
        print("hello!")


class DemoTestCase2(unittest.IsolatedAsyncioTestCase):
    def setUp(self):
        self.loop = asyncio.get_event_loop_policy().get_event_loop()

    async def test_demo(self):
        print("hello!")

if __name__ == "__main__":
    unittest.main()

see also #93896

 graingert@conscientious  testing310  ~/projects  python3.10 foo.py 
hello!
.hello!
.
----------------------------------------------------------------------
Ran 2 tests in 0.002s

OK

 graingert@conscientious  testing310  ~/projects  python3.11 foo.py 
hello!
.E
======================================================================
ERROR: test_demo (__main__.DemoTestCase2.test_demo)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.11/unittest/async_case.py", line 82, in _callSetUp
    self._asyncioTestContext.run(self.setUp)
  File "/home/graingert/projects/foo.py", line 14, in setUp
    self.loop = asyncio.get_event_loop_policy().get_event_loop()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/events.py", line 677, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'MainThread'.

----------------------------------------------------------------------
Ran 2 tests in 0.002s

FAILED (errors=1)

it looks like this is because asyncio.Runner calls asyncio.set_event_loop( in Runner.run

it looks like asyncio.set_event_loop is also being called an additional time per call to asyncio.Runner.run which will call get_child_watcher().attach_loop(loop) every call which causes the PidfdChildWatcher to blow away its callbacks

yep that's exactly what's happening here:

import asyncio
import sys

asyncio.set_child_watcher(asyncio.PidfdChildWatcher())

tasks = set()
pids = set()


async def subprocess():
    proc = await asyncio.create_subprocess_exec(
        sys.executable,
        "-c",
        "import time; print('hello!'); import time; time.sleep(2); print('goodbye')",
    )
    pids.add(proc.pid)
    print(await proc.communicate())


async def main():
    task = asyncio.create_task(subprocess())
    tasks.add(task)
    await asyncio.sleep(1)


async def main2():
    try:
        async with asyncio.timeout(3):
           await asyncio.gather(*tasks)
    except TimeoutError:
        import psutil
        for pid in pids:
            print(psutil.Process(pid))
    else:
        print("success!")

if __name__ == "__main__":
    with asyncio.Runner() as runner:
        runner.run(main())
        runner.run(main2())

output

hello!
goodbye
psutil.Process(pid=406749, name='python', status='zombie', started='10:13:40')

#95584 is vaguely related

The issue here is that asyncio.Runner calls set_event_loop too late and too often

In addition child watchers that use attach_loop are prone to unusual behaviour when the main thread switches loop

It looks like IsolatedAsyncioTestCase when the Pidfdchildwatcher is used will break a subprocess started in asyncSetUp

import asyncio
import sys
import unittest

import asyncio

asyncio.set_child_watcher(asyncio.PidfdChildWatcher())


def create_free_port():
    return 4  # chosen by a fair dice roll


class TestProc(unittest.IsolatedAsyncioTestCase):
    async def asyncSetUp(self):
        self.port = create_free_port()
        self.proc = await asyncio.create_subprocess_exec(
            sys.executable,
            "-c",  # more realistically this might be an http server or database
            f"import time; print('listening on {self.port}'); import time; time.sleep(2); print('goodbye')",
        )

    async def testProc(self):
        print(f"connecting to {self.port}")

    async def asyncTearDown(self):
        await self.proc.communicate()  # hangs forever on 3.11


if __name__ == "__main__":
    unittest.main()

on python3.10 this produces:

connecting to 4
listening on 4
goodbye
.
----------------------------------------------------------------------
Ran 1 test in 2.022s

OK

on python3.11 it hangs forever in await self.proc.communicate()

pythongh-95736: Fix event loop creation in IsolatedAsyncioTestCase (p…

[3.11] pythonGH-95736: fix IsolatedAsyncioTestCase to initialize Runn…

@graingert

@graingert I hope you will make aiohttp compatible with 3.11 regardless of whether this fix gets merged into 3.11. As a base dependency of many other packages it would block 3.11 support for those other packages.

@gvanrossum

@gvanrossum your question has, as always, helped me crystallize the trade off I chose by accident, and some other options that are available to aiohttp and I'm going to try to enumerate them so that aiohttp can make the right choice here

Whoops, reopening since the 3.11 backport didn’t get merged yet.

[3.11] GH-95736: fix IsolatedAsyncioTestCase to initialize Runner bef… (

Navigation Menu

Repository navigation

`IsolatedAsyncioTestCase` does not call `asyncio.set_event_loop` before `setUp` anymore, `asyncio.Runner`+`PidFdChildWatcher` leaves zombie processes #95736

18 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

IsolatedAsyncioTestCase does not call asyncio.set_event_loop before setUp anymore, asyncio.Runner+PidFdChildWatcher leaves zombie processes #95736

Description

Activity

graingert commented on Aug 6, 2022

graingert commented on Aug 6, 2022

graingert commented on Aug 6, 2022

graingert commented on Aug 8, 2022

graingert commented on Aug 10, 2022

graingert commented on Aug 11, 2022

18 remaining items

gvanrossum commented on Aug 17, 2022

graingert commented on Aug 17, 2022

gvanrossum commented on Aug 18, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`IsolatedAsyncioTestCase` does not call `asyncio.set_event_loop` before `setUp` anymore, `asyncio.Runner`+`PidFdChildWatcher` leaves zombie processes #95736