Repository navigation
Allow comm message to be processed during cell execution #65
Description
The use case: a Python function getMouse() that blocks waiting for the user to click on an SVG image that has been wired up to a trait to pass the x,y value back to Python. We would like getMouse() to return that x,y.
The implementation we are thinking of would allow comm messages to be processed during the cell execution. I think this is general to any comm_msg, but our goal is just the mouse widget. Here is a little version of the idea:
from ipywidgets import widgets
from IPython.display import display
i = widgets.IntText()
display(i)
i.value = 78
while i.value == 78:
passWe would like to be able to change the value in the displayed IntText (for example) and have the loop stop when the detected value changes. We understand that jupyter is not designed to do this, but we would like to replicate a pre-existing API in the notebook, getMouse().
Is it possible, while the cell is executing, to process any queued comm messages? Thinking of something like:
while i.value == 78:
kernel = get_ipython_kernel()
kernel.process_comm_msgs()Any help or alternative ideas welcomed!
Activity
minrk commentedon Oct 12, 2015
I think this can be handled by processing messages in a thread while running code in another. What complicates things is that comm messages come on the same queue as execution, but I think we can deal with that if we are careful. I've been designing the threaded IO in my head for a bit, because I think it would solve several problems with flushing stdout as well.
dsblank commentedon Oct 12, 2015
@minrk Thanks for the time on this issue. BTW, when we implemented our backend in C#, we execute the cell code in the background:
https://bitbucket.org/ipre/calico/src/master/Source/Calico/ZMQServer.cs?fileviewer=file-view-default#ZMQServer.cs-964
I thought that that was the way it was supposed to work. I don't remember any adverse effects, but I think we made sure that the kernel acted as if it were blocking (didn't accept another cell execution message, for example). I do remember testing parallel cell execution, which worked, if you keep the output cells associated with the execution. Although, that can be a confusing UI. It might be nice to have a magic that would allow cell code to run in background, and not block. No need to write specific code to have it run in a thread if it is already running in a thread.
AlexTugarev commentedon Apr 27, 2017
@minrk I'm glad, I found this issue, because I also need to handle
commmessages in background. I'm about to prototype a kernel based onIPythonKernelwhich processescommmessages in another thread. My plan is to use two queues forcommand other messages, which would be polled in different threads. Please share your ideas on the treaded IO, maybe it's helpful.Also I'd like to create a PR once it passes existing tests, and it would be great to receive feedback on this.
oeway commentedon May 16, 2020
I recently encountered the same issue, in the cell we run something like
await prompt("x=?")and the front end popup a dialog to get user input, once done, it sends a comm message to the backend. However, because the cell is running, the comm message cannot go through, it ended up with a dead state that we can only send an interrupt execution command to end it.@AlexTugarev do you have a chance to work on a solution? Two queues for comm and other messages sounds like a good idea.