Receiving messages from frontend when the main thread is busy

[govinda18]

trafficstars

Problem

Currently, the messages from frontend to kernel are received via he shell channel which is flushed only when the main thread is free. If there is a long process having the main thread occupied, there is no way to receive updates from the front end.

There was some discussion in https://github.com/jupyter-widgets/ipywidgets/issues/1349 related to this but the issue seems to be closed without reaching a conclusion.

I am not sure if it should be logged with jupyter_client, I am starting here as it is fairly easy to create a reproducer of this issue with a practical usecase using ipywidgets.

Consider, the following as a prototype for a data explorer tool:

import ipywidgets as widgets
import numpy as np
import time
from IPython.display import display

arr = np.array(range(1000))

# A widget to explore the data of the array
slider = widgets.IntRangeSlider(
    value=[0, 100],
    min=0,
    max=len(arr),
    step=1,
    description='Range Selector:',
    disabled=False,
    continuous_update=False,
    orientation='horizontal',
    readout=True,
    readout_format='d',
)
output = widgets.Output()

display(slider, output)

def on_change(change):
    with output:
        output.clear_output()
        st, en = change['new']
        new_arr = arr[st:en]
        print(f"Sum: {new_arr.sum()}")
        print(f"Mean: {new_arr.mean()}")
        print(f"Std: {new_arr.std()}")

slider.observe(on_change, names='value')

# An array processing function that takes 
# a long time
def process_array(arr):
    time.sleep(10)
    return arr.sum()


process_array(arr)

The usecase here is to explore the data while it is being processed. One of the solutions proposed in the issue above is to move the process_array in a thread so the main thread is free. This however is not a great solution as it requires the user to do some manual work at their end. Moreover, it is not always feasible either, say when the functions are not thread safe.

Proposed Solution

While I have not thought a much on how to solve this as my understanding of the jupyter messaging system is limited to the docs here, one way I think it can possibly be solved is by providing an on_recv handler for the ZMQStream used by the ipython kernel for receiving messages from the frontend.

Post that, we can may be try to process the received message in a separate thread.

Again, I am not sure of the complications involved here and if this is even a feasible/practical solution. The idea basically is to get a way to read the messages waiting in the shell channel.

[davidbrochart]

It might be solved at the kernel level. For instance, akernel is an asynchronous kernel that allows to process messages from the shell channel while a cell is running, provided that this cell is cooperative. What this means is that you can execute a cell that takes some time to compute a result, and at the same time another cell that can e.g. inspect the result being computed:

https://user-images.githubusercontent.com/4711805/170366595-6ffb00b5-5268-4c78-adfa-bc31e8727dc9.mp4

While akernel works in an async model of concurrency, the idea could be transposed to a threaded model of concurrency.

[davidbrochart]

The usecase here is to explore the data while it is being processed. One of the solutions proposed in the issue above is to move the process_array in a thread so the main thread is free. This however is not a great solution as it requires the user to do some manual work at their end. Moreover, it is not always feasible either, say when the functions are not thread safe.

The debugger can be used for that purpose. It has a variable explorer that can inspect variables even while the kernel is busy, because it uses the control channel which is handled in another thread.

[mlucool]

On a high level, we are looking for a way to make IPyWidgets do some interactions while the notebook is busy in another thread using ipykernel. One example use case is a table with 10M rows. If you want to page in data as you scroll, that breaks when some other cell is doing work.

I came across this, which I found interesting. It seems to work as I expect without ipywidgets but not with it.

from IPython.lib.backgroundjobs import BackgroundJobManager
jobs = BackgroundJobManager()
def foo():
    global x
    sleep(3)
    x = 5
    sleep(3)
    print('done')
    
jobs.new(foo)

# cell 2
x = 1
sleep(1)
x

# Cell 3
x = 1
sleep(7)
x

As expected, this weaves changes from foo and other cells.

So what if we tried this with ipywidgets?

import ipywidgets as widgets
import numpy as np
import time
from IPython.display import display
from IPython.lib.backgroundjobs import BackgroundJobManager
jobs = BackgroundJobManager()

arr = np.array(range(1000))

def foobar():
    sleep(5)
    # A widget to explore the data of the array
    slider = widgets.IntRangeSlider(
        value=[0, 100],
        min=0,
        max=len(arr),
        step=1,
        description='Range Selector:',
        disabled=False,
        continuous_update=False,
        orientation='horizontal',
        readout=True,
        readout_format='d',
    )
    output = widgets.Output()

    display(slider, output)

    def on_change(change):
        with output:
            output.clear_output()
            st, en = change['new']
            new_arr = arr[st:en]
            print(f"Sum: {new_arr.sum()}")
            print(f"Mean: {new_arr.mean()}")
            print(f"Std: {new_arr.std()}")

    slider.observe(on_change, names='value')
    
jobs.new(foobar)

# cell 2
# An array processing function that takes 
# a long time
def process_array(arr):
    time.sleep(5)
    s = arr.sum()
    time.sleep(5)
    return s

process_array(arr)

I suspect this is because some of the ipywidget code to listen to the comm is on the main thread.