From 4747a9a7700eb99dc8757bf259107f7993816cc5 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Mon, 27 Jun 2022 16:58:56 -0700 Subject: [PATCH 1/7] GH-90908: Document TaskGroup --- Doc/library/asyncio-task.rst | 49 ++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 7796e47d4674dc1..c0a713691b513d9 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -103,6 +103,29 @@ To actually run a coroutine, asyncio provides three main mechanisms: world finished at 17:14:34 +* The :class:`asyncio.TaskGroup` class provides a more modern + alternative to :func:`create_task`. + Using this API the last example becomes:: + + async def main(): + async with asyncio.TaskGroup() as tg: + task1 = tg.create_task( + say_after(1, 'hello')) + + task2 = tg.create_task( + say_after(2, 'world')) + + print(f"started at {time.strftime('%X')}") + + # The wait is implicit when the context manager exits. + + print(f"finished at {time.strftime('%X')}") + +The timing and output should be the same as for the previous version. + + .. versionchanged:: 3.11 + Added :class:`asyncio.TaskGroup`. + .. _asyncio-awaitables: @@ -223,6 +246,11 @@ Creating Tasks :exc:`RuntimeError` is raised if there is no running loop in current thread. + .. note:: + + :meth:`asyncio.TaskGroup.create_task` is a newer alternative + that allows for convenient waiting for a group of related tasks. + .. important:: Save a reference to the result of this function, to avoid @@ -254,6 +282,27 @@ Creating Tasks Added the *context* parameter. +Task Groups +=========== + +Task groups combine a task creation API with a convenient +and reliable API to wait for completion of all tasks in the group. + +.. class:: TaskGroup() + + An :ref:`asynchronous context manager ` + holding a group of tasks. + Tasks can be added to the group using the :meth:`create_task` method. + All tasks are awaited when the context manager exits. + + .. versionadded:: 3.11 + + .. method:: create_tasks(coro, *, name=None, context=None) + + Create a task in this task group. + The API is the same as the :func:`asyncio.create_task` function. + + Sleeping ======== From 293c46b9c06e0a61ff0fa86d6ceb8894572250d8 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Mon, 27 Jun 2022 20:21:41 -0700 Subject: [PATCH 2/7] Link to example --- Doc/library/asyncio-task.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index c0a713691b513d9..85030a945ccdf3b 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -287,6 +287,7 @@ Task Groups Task groups combine a task creation API with a convenient and reliable API to wait for completion of all tasks in the group. +For an example see the last bullet under :ref:`coroutine`. .. class:: TaskGroup() From 8f5f5ec5822c66a403aaf5cd4863c3be21471662 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Mon, 27 Jun 2022 20:28:19 -0700 Subject: [PATCH 3/7] Mention task groups in what's new 3.11 --- Doc/whatsnew/3.11.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 256d5822e5196ee..9af55ccd20e9191 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -443,6 +443,10 @@ asyncio the asyncio library. (Contributed by Yves Duprat and Andrew Svetlov in :gh:`87518`.) +* Add :class:`~asyncio.TaskGroup` class, an asynchronous context manager + holding a group of tasks that will wait for all of them upon exit. + (Contributed by Yury Seliganov and others.) + datetime -------- From 57bef418421a4476e3dc98c5787d20377ddea2dc Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Tue, 28 Jun 2022 17:47:53 -0700 Subject: [PATCH 4/7] Describe what happens when tasks fail --- Doc/library/asyncio-task.rst | 60 +++++++++++++++++++++++++++++++++--- 1 file changed, 55 insertions(+), 5 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 85030a945ccdf3b..3dbede2bb7a97a2 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -286,22 +286,72 @@ Task Groups =========== Task groups combine a task creation API with a convenient -and reliable API to wait for completion of all tasks in the group. -For an example see the last bullet under :ref:`coroutine`. +and reliable API to wait for all tasks in the group to finish. + +Example:: + + async def main(): + async with asyncio.TaskGroup() as tg: + task1 = tg.create_task(some_coro(...)) + task2 = tg.create_task(another_coro(...)) + print("Both tasks have completed now.") + +The ``async with`` statement will wait for all tasks in the group to finish. +While waiting, new tasks may still be added to the group +(for example, by passing ``tg`` into one of the coroutines +and calling ``tg.create_task()`` in that coroutine). +Once the last task has finished and the ``async with`` block is exited, +no new tasks may be added to the group. + +When one of the tasks belonging to the group fails +with an exception other than :exc:`asyncio.CancelledError`, +the remaining tasks in the group are cancelled. +This cancellation of the remaining tasks is done only +the first time any task fails (excluding :exc:`asyncio.CancelledError`). +At this point, if the body of the ``async with`` statement is still active +(i.e., ``__aexit__`` hasn't been called yet), +the task directly containing the ``async with`` statement is also cancelled. +The resulting :exc:`asyncio.CancelledError` will interrupt an ``await``, +but it will not bubble out of the containing ``async with`` statement. + +Once all tasks have finished, if any tasks have failed +with an exception other than :exc:`asyncio.CancelledError`, +those exceptions are combined in an +:exc:`ExceptionGroup` or :exc:`BaseExceptionGroup` +(as appropriate, see their documentation) +which is then raised. + +Two base exceptions are treated specially: +If any task fails with :exc:`KeyboardInterrupt` or :exc:`SystemExit`, +the task group still cancels the remaining tasks and waits for them, +but then the initial :exc:`KeyboardInterrupt` or :exc:`SystemExit` +is re-raised instead of :exc:`ExceptionGroup` or :exc:`BaseExceptionGroup`. + +If the body of the ``async with`` statement exits with an exception +(so ``__aexit__`` is called with an exception set), +this is treated the same as if one of the tasks failed: +the remaining tasks are cancelled and then waited for, +and non-cancellation exceptions are grouped into a +(base) exception group and raised. +The exception passed into ``__aexit__``, +unless it is :exc:`asyncio.CancelledError`, +is also included in the exception group. +The same special case is made for +:exc:`KeyboardInterrupt` and :exc:`SystemExit` here. .. class:: TaskGroup() An :ref:`asynchronous context manager ` holding a group of tasks. - Tasks can be added to the group using the :meth:`create_task` method. + Tasks can be added to the group using :meth:`create_task`. All tasks are awaited when the context manager exits. .. versionadded:: 3.11 - .. method:: create_tasks(coro, *, name=None, context=None) + .. method:: create_task(coro, *, name=None, context=None) Create a task in this task group. - The API is the same as the :func:`asyncio.create_task` function. + The signature matches that of :func:`asyncio.create_task`. Sleeping From 240bf48e342fd6cad3e1993a000af9574d52e080 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Thu, 30 Jun 2022 08:48:52 -0700 Subject: [PATCH 5/7] CAM's code review Co-authored-by: CAM Gerlach --- Doc/library/asyncio-task.rst | 28 +++++++++++++--------------- Doc/whatsnew/3.11.rst | 3 ++- 2 files changed, 15 insertions(+), 16 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 3dbede2bb7a97a2..d7710b8b378f549 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -105,7 +105,7 @@ To actually run a coroutine, asyncio provides three main mechanisms: * The :class:`asyncio.TaskGroup` class provides a more modern alternative to :func:`create_task`. - Using this API the last example becomes:: + Using this API, the last example becomes:: async def main(): async with asyncio.TaskGroup() as tg: @@ -121,10 +121,10 @@ To actually run a coroutine, asyncio provides three main mechanisms: print(f"finished at {time.strftime('%X')}") -The timing and output should be the same as for the previous version. + The timing and output should be the same as for the previous version. - .. versionchanged:: 3.11 - Added :class:`asyncio.TaskGroup`. + .. versionadded:: 3.11 + :class:`asyncio.TaskGroup`. .. _asyncio-awaitables: @@ -286,7 +286,7 @@ Task Groups =========== Task groups combine a task creation API with a convenient -and reliable API to wait for all tasks in the group to finish. +and reliable way to wait for all tasks in the group to finish. Example:: @@ -303,13 +303,11 @@ and calling ``tg.create_task()`` in that coroutine). Once the last task has finished and the ``async with`` block is exited, no new tasks may be added to the group. -When one of the tasks belonging to the group fails +The first time any of the tasks belonging to the group fails with an exception other than :exc:`asyncio.CancelledError`, the remaining tasks in the group are cancelled. -This cancellation of the remaining tasks is done only -the first time any task fails (excluding :exc:`asyncio.CancelledError`). At this point, if the body of the ``async with`` statement is still active -(i.e., ``__aexit__`` hasn't been called yet), +(i.e., :meth:`~object.__aexit__` hasn't been called yet), the task directly containing the ``async with`` statement is also cancelled. The resulting :exc:`asyncio.CancelledError` will interrupt an ``await``, but it will not bubble out of the containing ``async with`` statement. @@ -318,7 +316,7 @@ Once all tasks have finished, if any tasks have failed with an exception other than :exc:`asyncio.CancelledError`, those exceptions are combined in an :exc:`ExceptionGroup` or :exc:`BaseExceptionGroup` -(as appropriate, see their documentation) +(as appropriate; see their documentation) which is then raised. Two base exceptions are treated specially: @@ -328,16 +326,16 @@ but then the initial :exc:`KeyboardInterrupt` or :exc:`SystemExit` is re-raised instead of :exc:`ExceptionGroup` or :exc:`BaseExceptionGroup`. If the body of the ``async with`` statement exits with an exception -(so ``__aexit__`` is called with an exception set), +(so :meth:`~object.__aexit__` is called with an exception set), this is treated the same as if one of the tasks failed: the remaining tasks are cancelled and then waited for, -and non-cancellation exceptions are grouped into a -(base) exception group and raised. -The exception passed into ``__aexit__``, +and non-cancellation exceptions are grouped into an +exception group and raised. +The exception passed into :meth:`~object.__aexit__`, unless it is :exc:`asyncio.CancelledError`, is also included in the exception group. The same special case is made for -:exc:`KeyboardInterrupt` and :exc:`SystemExit` here. +:exc:`KeyboardInterrupt` and :exc:`SystemExit` as in the previous paragraph. .. class:: TaskGroup() diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 9af55ccd20e9191..9eafd6da13fa5d8 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -443,7 +443,8 @@ asyncio the asyncio library. (Contributed by Yves Duprat and Andrew Svetlov in :gh:`87518`.) -* Add :class:`~asyncio.TaskGroup` class, an asynchronous context manager +* Add :class:`~asyncio.TaskGroup` class, + an :ref:`asynchronous context manager ` holding a group of tasks that will wait for all of them upon exit. (Contributed by Yury Seliganov and others.) From 1ec56fb4f572d74b1a9b5fd6d0242dbca9c5e83c Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Thu, 30 Jun 2022 09:01:43 -0700 Subject: [PATCH 6/7] Add a note recommending TaskGroup to gather Also remove a redundant versionchanged directive. --- Doc/library/asyncio-task.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index d7710b8b378f549..4f9a50b288ce650 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -425,8 +425,9 @@ Running Tasks Concurrently cancellation of one submitted Task/Future to cause other Tasks/Futures to be cancelled. - .. versionchanged:: 3.10 - Removed the *loop* parameter. + .. note:: + A more modern way to create and run tasks concurrently and + wait for their completion is :class:`asyncio.TaskGroup`. .. _asyncio_example_gather: From 812605751bee4fad8a2021705ff5fe9ea9e10e49 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Thu, 30 Jun 2022 09:04:38 -0700 Subject: [PATCH 7/7] Move 'class TaskGroup' up in its section --- Doc/library/asyncio-task.rst | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index 4f9a50b288ce650..9b76548433b6a75 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -288,6 +288,20 @@ Task Groups Task groups combine a task creation API with a convenient and reliable way to wait for all tasks in the group to finish. +.. class:: TaskGroup() + + An :ref:`asynchronous context manager ` + holding a group of tasks. + Tasks can be added to the group using :meth:`create_task`. + All tasks are awaited when the context manager exits. + + .. versionadded:: 3.11 + + .. method:: create_task(coro, *, name=None, context=None) + + Create a task in this task group. + The signature matches that of :func:`asyncio.create_task`. + Example:: async def main(): @@ -337,20 +351,6 @@ is also included in the exception group. The same special case is made for :exc:`KeyboardInterrupt` and :exc:`SystemExit` as in the previous paragraph. -.. class:: TaskGroup() - - An :ref:`asynchronous context manager ` - holding a group of tasks. - Tasks can be added to the group using :meth:`create_task`. - All tasks are awaited when the context manager exits. - - .. versionadded:: 3.11 - - .. method:: create_task(coro, *, name=None, context=None) - - Create a task in this task group. - The signature matches that of :func:`asyncio.create_task`. - Sleeping ========