Skip to content

shield() documentation should mention user needs to keep reference to the task #94972

Description

@graingert

Documentation

asyncio will only keep weak references to alive tasks (in _all_tasks). If a user does not keep a reference to a task and the task is not currently executing or sleeping, the user may get "Task was destroyed but it is pending!".

see also #88831

shield has the same issue and the documentation currently recommends the incorrect usage

Activity

  1. changed the title [-]sheild() documentation should mention user needs to keep reference to the task[/-] [+]shield() documentation should mention user needs to keep reference to the task[/+] on Jul 18, 2022
  2. kumaraditya303 commented on Jul 22, 2022

    @kumaraditya303
    Contributor

    Can you provide reproducible code where this happens?

  3. graingert commented on Jul 22, 2022

    @graingert
    ContributorAuthor

    See also #93297

  4. graingert commented on Jul 22, 2022

    @graingert
    ContributorAuthor

    @kumaraditya303 here's the repro on python3.10:

    import weakref
    import asyncio
    import gc
    
    import logging
    
    
    logger = logging.getLogger(__name__)
    
    
    async def async_fn():
        try:
            await asyncio.get_running_loop().create_future()
        except BaseException:
            logger.exception("closed!")
            raise
    
    
    async def amain():
        weak_task = weakref.ref(asyncio.create_task(async_fn()))
        await asyncio.sleep(0.01)
        gc.collect()
        print(weak_task())
    
    
    async def sheild_gc():
        strong_shielded_fut = asyncio.shield(async_fn())
        await asyncio.sleep(0)
        asyncio.current_task().cancel()
        try:
            await strong_shielded_fut
        except asyncio.CancelledError:
            pass
    
        gc.collect()
    
    
    asyncio.run(amain())
    print("======================")
    asyncio.run(sheild_gc())

    output:

    closed!
    Traceback (most recent call last):
      File "/home/graingert/projects/foo.py", line 14, in async_fn
        await asyncio.get_running_loop().create_future()
    GeneratorExit
    Task was destroyed but it is pending!
    task: <Task pending name='Task-2' coro=<async_fn() done, defined at /home/graingert/projects/foo.py:12> wait_for=<Future pending cb=[Task.task_wakeup()]>>
    None
    ======================
    closed!
    Traceback (most recent call last):
      File "/home/graingert/projects/foo.py", line 14, in async_fn
        await asyncio.get_running_loop().create_future()
    GeneratorExit
    Task was destroyed but it is pending!
    task: <Task pending name='Task-6' coro=<async_fn() done, defined at /home/graingert/projects/foo.py:12> wait_for=<Future pending cb=[Task.task_wakeup()]>>
    
  5. gvanrossum commented on Aug 24, 2022

    @gvanrossum
    Member

    Are you too busy to submit a PR that updates the docs?

  6. graingert commented on Aug 24, 2022

    @graingert
    ContributorAuthor

    There's one linked here #94973

  7. gvanrossum commented on Aug 29, 2022

    @gvanrossum
    Member

    I think I get the same output on 3.11. Question, is import objgraph needed in the repro? (I had to delete it, no idea what it is.)

  8. added a commit that references this issue on Sep 10, 2022
  9. added 4 commits that reference this issue on Sep 10, 2022
  10. graingert commented on Sep 12, 2022

    @graingert
    ContributorAuthor

    I think I get the same output on 3.11. Question, is import objgraph needed in the repro? (I had to delete it, no idea what it is.)

    Left in by accident. It's not needed to cause the error but I used it to help build it

  11. added a commit that references this issue on Dec 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions