ipywidgets cannot render when connected to a kernel of another notebook

[nulinspiratie]

trafficstars

I am having issues displaying ipywidgets when two notebooks share the same kernel. The original notebook (which instantiated the kernel) is still able to display widgets. However, if I connect another notebook to the same kernel, the second notebook is unable to display any widgets. Instead, an error message is displayed (see below for details). Oddly enough, the original notebook can still display widgets without any issues.

The issue is somewhat related to #1218 and #1881 although the issues are fairly old, and it sounds as though it should be possible to use ipywidgets when a kernel is shared by multiple frontends.

Is this a known issue, and is there any solution to this?

Steps to reproduce:

1. Create notebook1.ipynb and initialize kernel.
2. Create notebook2.ipynb and initialize kernel.
3. Change kernel of notebook2.ipynb to that of notebook1.ipynb
4. Run the following code on notebook2.ipynb:

from ipywidgets.widgets import Button
Button()

Expected result:

An ipywidget button appears

Actual results

Output shows Error displaying widget: model not found along with the following error message in the console:

System specs:

OS Windows 10 python 3.7 ipython 7.9.0 ipywidgets 7.5.1 jupyter 1.0.0 jupyter-client 5.3.4 jupyter-console 6.0.0 jupyter-core 4.6.1 jupyterlab 1.2.3 jupyterlab-server 1.0.6 NodeJS 12.13.0

$ jupyter labextension list JupyterLab v1.2.3 Known labextensions: app dir: c:\programdata\anaconda3\share\jupyter\lab @jupyter-widgets/jupyterlab-manager v1.1.0 enabled ok @jupyter-widgets/jupyterlab-sidecar v0.4.0 enabled ok bqplot v0.5.1 enabled ok jupyter-matplotlib v0.4.2 enabled ok

[nulinspiratie]

Is there anyone who knows if ipywidgets are or are not meant to work with a shared kernel?

If they are, any idea if the issue outlined above might have a simple fix, or that it hits at a fundamental ipywidgets limitation? If someone can point me in the right direction it would help me to look further into it

[jasongrout]

Unfortunately, the underlying comm message spec has some flaws when you have multiple clients talking to the same kernel, so no, widgets can't be reliably used on multiple simultaneous notebooks. I would love to see these flaws fixed, but it is going to take a change to the Jupyter protocol. See https://github.com/jupyter/jupyter_client/issues/263 for more details.

[jasongrout]

One shortcut that we have built in to JLab is that if you have a single notebook and make two views of that same notebook, or "Create new output view" of some of the cells, it will use the same connection and same underlying widget models, and widgets will work in both tabs.

[nulinspiratie]

@jasongrout I have looked a bit more into this issue, and from what I can tell the underlying issue is that each notebook has its own WidgetManager. However, when a new widget is created (regardless of the notebook), it is always registered with the WidgetManager whose notebook started the shared kernel.

As a test, I manually updated WidgetManager._models of the second notebook to equal that of the first notebook (using Chrome DevTools). After doing so, the second notebook had no problems whatsoever displaying the widget. I'm therefore thinking, is there a way for a WidgetManager to check if there are other WidgetManagers that share the same kernel, and if so, broadcast the creation of a new widget? I think this would solve the problem.

I can currently only think of hacky solutions, would there be a straightforward proper fix?

[jasongrout]

is there a way for a WidgetManager to check if there are other WidgetManagers that share the same kernel, and if so, broadcast the creation of a new widget? I think this would solve the problem.

Or perhaps use a single widget manager per kernel, rather than a single widget manager per kernel connection (i.e., each document here has a separate kernel connection, but to the same kernel). I think that could maybe work, but we'd have to check ramifications since the widget manager also deals with the notebook, for example state is restored from the notebook document too, and having two different notebook documents share the same kernel manager would complicate that.

[jasongrout]

Setting as future as an idea to explore when someone has time. If you want to explore these ideas in a PR, we can help you get started.

[nvaytet]

Hi, is there any news on this? Thanks.

[maartenbreddels]

Yes, version 8.0 by default will echo messages, which will support multiple clients (added in https://github.com/jupyter-widgets/ipywidgets/pull/3195 )

However, I am not 100% sure it addresses the original issue of this issue. Can you try?

[saroop-samra]

Hi, we are building our product on top of ipywidgets and this is adversely affecting our user experience.

If you create a new notebook in Jupyter Notebooks, then sometimes, Jupyter Notebooks automatically assigns an existing kernel (i.e. no pop up appears to select a new kernel), at which point the widget callbacks with ipywidgets will not work in the notebook (although the rendering seems to work but UI events do not).

This is the simple example I replicated the issue with:

import ipywidgets as widgets
from IPython.display import display
button = widgets.Button(description="Click Me!")
output = widgets.Output()
display(button, output)
def on_button_clicked(b):
    with output:
        print("Button clicked.")
button.on_click(on_button_clicked)

You can simulate the issue by manually sharing a kernel between two notebooks that are running the above code.

The issue seems to be related to the usage of shared kernels in Jupyter Notebooks.

I am hoping this can get resolved in a future ipywidgets release. Note, I tested this in 8.1.5 and can reproduce the issue. If there are new builds that are available I am happy to test and provide feedback.

[mherrmann3]

However, I am not 100% sure it addresses the original issue of this issue. Can you try?

Unfortunately not; I'm using the most recent ipywidgets version (v8.1.7).

The behavior is like this:

1. upon loading the "main" notebook with the widgets, all widgets render well;
2. after connecting another (empty) notebook to that kernel, widgets in the main notebook are still shown;
3. after refreshing (F5) the main notebook, the widgets disappear (initially showing "Loading widget...", then "Error displaying widget: model not found".