Skip to content

Do not fire pre_execute/post_execute for comms dispatched concurrently during a cell - #1588

Open
Dikshant-Neupane wants to merge 2 commits into
ipython:mainfrom
Dikshant-Neupane:fix-1564-comm-post-execute
Open

Dikshant-Neupane wants to merge 2 commits into
ipython:mainfrom
Dikshant-Neupane:fix-1564-comm-post-execute

Conversation

@Dikshant-Neupane

Copy link
Copy Markdown

Closes #1564

When a comm message arrives while an async cell is running, the comm is dispatched concurrently while the cell holds the shell lock, and pre_execute/post_execute fire for the comm callback mid-cell. Any output a post_execute hook publishes is then parented to the comm_msg instead of the cell's execute_request, so frontends drop it, and the hook's queue is drained before the cell's own post_execute fires.

The sync-cell case was already fixed by #1565; this covers the remaining async-cell case, with and without a subshell.

Fix: a context variable is set in the shell_main concurrent-dispatch branch only, and ipykernel's BaseComm.handle_msg override skips the two events when it is set. Comms handled while no cell is running keep the previous behaviour. The approach follows the discussion in the issue.

Repro (issue's reproducer, on main f87d0e0):

  • Sync cell + subshell: passed before and after (fixed by Allow async cell to process Comm messages #1565).
  • Async cell + subshell: failed before (display parented to comm_msg, post_execute at about 1s mid-cell), passes after.
  • Async cell, no subshell: failed before, passes after.

Tests:

  • Full suite: 223 passed, 28 skipped, 0 failed (coverage gate passed).
  • New regression tests (unit in tests/test_comm.py, live-kernel in tests/test_subshells.py, parametrized subshell on/off): fail without the fix, pass with it.

…y during a cell

During an async cell, comm messages are dispatched concurrently while the cell holds the shell lock. Firing pre_execute/post_execute for those callbacks parents any hook output to the comm_msg instead of the cell's execute_request, so frontends drop it and the hook's queue is drained before the cell's own post_execute fires (see ipython#1564). A context variable marks the concurrent dispatch, and BaseComm.handle_msg skips the two events only there; comms handled while no cell is running keep the previous behaviour.
The annotation was evaluated at import time, so environments with a comm version that has no base_comm.MessageType failed to import the module. Quoting it makes the evaluation lazy; the runtime behaviour is unchanged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

post_execute fires during comm_msg handling on a subshell, so output published from the hook is parented to the comm_msg and dropped by frontends

1 participant