Repository navigation
asyncio loop.add_signal_handler() may not behave as expected #83946
Description
| BPO | 39765 |
|---|---|
| Nosy | @asvetlov, @1st1, @rogerdahl |
| PRs |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2020-02-26.20:46:52.392>
labels = ['type-bug', '3.8', 'expert-asyncio']
title = 'asyncio loop.add_signal_handler() may not behave as expected'
updated_at = <Date 2020-07-05.08:32:00.516>
user = 'https://github.com/rogerdahl'bugs.python.org fields:
activity = <Date 2020-07-05.08:32:00.516>
actor = 'tiposchi'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2020-02-26.20:46:52.392>
creator = 'rogerdahl'
dependencies = []
files = []
hgrepos = []
issue_num = 39765
keywords = ['patch']
message_count = 6.0
messages = ['362734', '362747', '362748', '362750', '362830', '373018']
nosy_count = 4.0
nosy_names = ['asvetlov', 'yselivanov', 'tiposchi', 'rogerdahl']
pr_nums = ['18664']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39765'
versions = ['Python 3.8']
Activity
rogerdahl commentedon Feb 26, 2020
This is a ticket to document two ways in which the behavior of loop.set_signal_handler() may not match what the user expects.
First, callbacks to handlers registered with loop.set_signal_handler() may be significantly delayed. I have a program where I've encountered delays of up to maybe a minute or so between hitting Ctrl-C and getting the callback for the SIGINT handler. During this time, the program works through queued aiohttp tasks. Though it's possible to have delays in callbacks for events set with signal.signal(), I haven't personally seen that, and I think that's the case for most users. So I think this point should be included in the docs.
Second, set_signal_handler() silently and implicitly removes corresponding handlers set with signal.signal(). Though perhaps logical, this potentially removes a "fast" handler and replaces it with a "slow" one. I think this should be documented as well.
asvetlov commentedon Feb 26, 2020
I'm sorry. but loop.set_signal_handler() method doesn't exist.
asvetlov commentedon Feb 26, 2020
Assuming you mean loop.add_signal_handler() method, I would say that a minute-long delay is a sign of long blocking calls in your program.
Typical asyncio program blocks the loop for 1-10 milliseconds at most, and signal handlers are called with this delay.
This is not your case as I see.
[-]asyncio loop.set_signal_handler() may not behave as expected[/-][+]asyncio loop.add_signal_handler() may not behave as expected[/+][-]asyncio loop.set_signal_handler() may not behave as expected[/-][+]asyncio loop.add_signal_handler() may not behave as expected[/+]rogerdahl commentedon Feb 26, 2020
Oops, fixed.
During the delay for the callback to occur, I see my program logging work performed on other asyncio tasks and receiving timer events from another thread. So, not long blocking?
asvetlov commentedon Feb 27, 2020
Other threads are not related.
Speaking about the main thread that runs an event loop with the signal handler installed -- the handler is executed along with other async code.
If the loop executes some user-provides async task -- it executes the signal handler as well if needed.
This is why I suspect that the discussed program is not completely correct.
tiposchi commentedon Jul 5, 2020
This looks related to: https://bugs.python.org/issue32443
kumaraditya303 commentedon Oct 9, 2022
When signal handlers are called is not guarantied, the only guarantee is that it will be called if the signal is received. It is an implementation detail e.g. if C code handles pending calls and checks for signal etc. I am -1 on adding anything regarding these implementation details, user code should not rely on any of this. This issue is most likely a bug in your code rather than in
asyncio.gvanrossum commentedon Oct 9, 2022
@rogerdahl Have you analyzed your program more? If you can show a small demo that exhibits this behavior we'd be happy to have a look.
rogerdahl commentedon Oct 21, 2022
@gvanrossum Please leave this open for a bit and I'll try to find some time to see what the current status is.
arhadthedev commentedon Mar 4, 2023
@rogerdahl ping?
rogerdahl commentedon Mar 10, 2023
@arhadthedev Thank you for following up. I've done some tests now and I'm unable to reproduce this. Please go ahead and close the ticket. I'll reopen it with code to reproduce if I encounter the issue again.
Just speculating, I think the problem might have been an interplay between signal handlers in my program and in aiohttp, which I was using heavily.
In my test, I set up an asyncio web server with delays in it to simulate slow responses, and an aiohttp based client that registered a signal handler and then hit the server with 1000 concurrent requests. The signal handler was still invoked instantly.
aiohttp registers its own signal handlers, including for SIGINT, which I also registered for. There's a comment in the signal handler setup in aiohttp that mentions workarounds for issues with Gunicorn, so maybe they were doing something odd there.