Repository navigation
Documentation enhancement proposal: explainer for asyncio #137026
Description
Documentation
I've used Python's asyncio a couple times now, but never really felt confident in my mental model of how it fundamentally works and therefore how I can best leverage it. The official docs provide decent documentation for each specific function in the package, but, in my opinion, lack a cohesive overview of the systems design and architecture. Something that could help the user understand the why and how behind the recommended patterns. And a way to help the user make informed decisions about which tool in the asyncio toolkit they ought to grab, or to recognize when asyncio is the entirely wrong toolkit.
I spent a long time digging into the internals and then decided to take a stab at filling that perceived gap by writing a fairly thorough long-form article: A conceptual overview of asyncio.
I also submitted the article to HackerNews where it got some traction: https://news.ycombinator.com/item?id=44638710
I imagine there's a few ways forward here.
- Linking to the Github article from the asyncio docs
- Integrating the article directly into the Python docs (along with stylistic & content modifications to match)
- Y'all decide the article's mediocre at best and you don't want it merged. (Fair enough!).
Either way, let me know what y'all figure makes the most sense. :)
Activity
StanFromIreland commentedon Jul 22, 2025
I am afraid it will not be possible to merge your article in the current state. From my understanding of your post, a HOWTO for asyncio would solve your problem?
anordin95 commentedon Jul 22, 2025
I don't think a HOWTO would help in this case. In the Diataxis framework that the Python docs prefer, the aim of my article, and the thing I feel is missing from asyncio's docs is an explainer/explanation component. Something that can provide a more thorough, fundamental understanding.
StanFromIreland commentedon Jul 22, 2025
That would be the purpose of the HOWTO, like the existing ones for other modules, no? It can then be linked in the main doc, which contains the technical documentation?
AA-Turner commentedon Jul 22, 2025
Thank you for the proposal Alex. We have a mini tutorial in https://docs.python.org/dev/library/asyncio-dev.html, but it could be expanded.
We'd be willing to take more explainer content, but it would need to be merged in small chunks, rather than all 6 pages at once. You would also need to rewrite the content to use reStructuredText and fit into the Python documentation.
I think more documentation on asyncio, and concurrency in general, is certainly welcome -- we now have regular threads, multiprocessing, subinterpreters, async, free threading, and perhaps more (greenlet?).
cc @kumaraditya303 for thoughts.
A
anordin95 commentedon Jul 22, 2025
@StanFromIreland, I feel we may be talking past each other a bit here. I'm working with the below definitions. And based on those, I feel my article falls much more in the explanatory realm. And, such explanatory information is what I feel is missing from asyncio.
Taken from: https://devguide.python.org/documentation/style-guide/#diataxis
anordin95 commentedon Jul 22, 2025
@AA-Turner -- gotcha, sounds good! I wonder if you could also provide a rough gauge or check-in on whether the content of this specific article would fit well with what's desired for such explainers or if it's currently very far off of the kind of prose that's commonly accepted.
AA-Turner commentedon Jul 22, 2025
I've asked our
asyncioexperts for a second opinion on the technical merits of the content. Based on brief skim of the structure, I think it looks reasonable. I would aim to make some of the code blocks shorter, with explanations interspersed of what is going on. For example, in part 2 you have two blocks that are over 20 lines long -- this is a few full screens on a mobile browser. I would personally make the prose less folksy (e.g. 'regular 'ol function' is cute, but distracts from the content). The metaphor of a conductor/military planner/etc is a nice one, though.I'm not sure from a pedagodical perspective if it makes sense to introduce the higher level (more frequently used) concepts such as tasks before the low-level building blocks (coroutines). Currently, you go low-level to high-level. Perhaps @treyhunner or @nedbat could opine?
Likewise, the "when to use asyncio" I think could be more prominent than in the conclusion -- it could even deserve a standalone section (or as part of a general 'what concurrency do I want?' FAQ). The 'design choices' section would probably stay in your article rather than moving accross. I like the variety of examples, which is always a good thing to have.
A
anordin95 commentedon Jul 22, 2025
Mmm, sounds good! Though, I do think whether coroutines are low-level is somewhat up for debate. For example, in the current asyncio docs they're listed under the high level API and introduced just before tasks. For reference, I did roughly intend to go from high-level to low-level, but you are right that there are some cases where I don't, like the event-loop.
AA-Turner commentedon Jul 26, 2025
Also cross-referencing #79012 ("Asyncio Tutorial"), and #9748 as a previous attempt, but was too large to review. We've also had feedback that the detailed internals of how coroutines & await work shouldn't be closely tied to documentation for
asyncio.There's definite interest in this explainer-style prose, but I think past attempts have tried to do too much at once. I would suggest splitting the effort into a series of PRs, perhaps initially adding a small article and then expanding it incrementally. Feel free to reuse any of the content from the closed PRs, too -- as far as I can see, these were closed due to process problems, not because the content was bad.
A
anordin95 commentedon Jul 26, 2025
Right on! I've mostly got the content transferred to reST and have been cutting down on chaff in the code examples to reduce their size. The current article is roughly 3500 words (or 7 pages single spaced). I've still got to do a few passes for grammar, consistency, style, etc.
anordin95 commentedon Jul 29, 2025
@AA-Turner
Thank you for the feedback btw! I incorporated some of your ideas like titling the section "Which concurrency do I want", making it more prominent, and reducing the size of many code-blocks and incorporating more plaintext descriptions of those blocks. I've just put the PR up here: #137215
gh-137026: Add an explainer guide for asyncio (GH-137215)
pythongh-137026: Add an explainer guide for asyncio (pythonGH-137215)
pythongh-137026: Add an explainer guide for asyncio (pythonGH-137215)
[3.13] gh-137026: Add an explainer guide for asyncio (GH-137215) (#13…
[3.14] gh-137026: Add an explainer guide for asyncio (GH-137215) (#13…
pythongh-137026: Add an explainer guide for asyncio (pythonGH-137215)
[3.14] pythongh-137026: Add an explainer guide for asyncio (pythonGH-…