Support for jupyterlite

[Kirill888]

Currently code assumes that IPython is using ZMQ based kernel. I think it should be possible to support pyolite as well.

1. ~Remove compulsory dependency on zmq and tornado (imports and packaging)~ [DONE]
2. Tweak replay function to behave differently based on underlying kernel used by IPython

[Kirill888]

looks like jupyterlite forces message ordering in javascript land, so no new messages will arrive into python until cell execution is finished:

      const processMsg = async (msg: KernelMessage.IMessage) => {
        await mutex.runExclusive(async () => {
          await kernel.handleMessage(msg);
        });
      };

https://github.com/jupyterlite/jupyterlite/blob/feffad32b8efb402d0a76218c46b9746e37f75f4/packages/kernel/src/kernels.ts#L64-L68

[drazellan]

Hello and first thank you for your work I try to use your code in jupyter lite

import piplite
await piplite.install("ipywidgets")
await piplite.install("jupyter_ui_poll")

from ipywidgets import IntSlider, Output
import time
from jupyter_ui_poll import ui_events

slider = IntSlider()
out = Output()
display(slider)
display(out)


with ui_events() as poll:
    while slider.value<50 :
        poll(10)          # React to UI events (upto 10 at a time)
        print('.', end='')
        time.sleep(0.1)
print('done')

and i got the following error :

--------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In [8], line 11
      7 display(slider)
      8 display(out)
---> 11 with ui_events() as poll:
     12     while slider.value<50 :
     13         poll(10)          # React to UI events (upto 10 at a time)

File /lib/python3.10/site-packages/jupyter_ui_poll/_poll.py:234, in ui_events()
    207 def ui_events():
    208     """
    209     Gives you a function you can call to process UI events while running a long
    210     task inside a Jupyter cell.
   (...)
    232        cell execution is finished.
    233     """
--> 234     return KernelWrapper.get()

File /lib/python3.10/site-packages/jupyter_ui_poll/_poll.py:145, in KernelWrapper.get()
    142 @staticmethod
    143 def get() -> "KernelWrapper":
    144     if KernelWrapper._current is None:
--> 145         KernelWrapper._current = KernelWrapper(
    146             get_ipython(), asyncio.get_event_loop()
    147         )
    148     return KernelWrapper._current

File /lib/python3.10/site-packages/jupyter_ui_poll/_poll.py:40, in KernelWrapper.__init__(self, shell, loop)
     37 self._kernel = kernel
     38 self._loop = loop
     39 self._original_parent = (
---> 40     kernel._parent_ident,
     41     kernel.get_parent()  # ipykernel 6+
     42     if hasattr(kernel, "get_parent")
     43     else kernel._parent_header,  # ipykernel < 6
     44 )
     45 self._events: List[Tuple[Any, Any, Any]] = []
     46 self._backup_execute_request = kernel.shell_handlers["execute_request"]

AttributeError: 'Pyolite' object has no attribute '_parent_ident'

I also try the trick with do_one_interation() but it doesn't work either.

AttributeError: 'Pyolite' object has no attribute 'do_one_iteration'

Do you have any idea to make it work? Thanks in advance

[Kirill888]

Currently this library only works with ipykernel, it should probably detect other kernels and either raise an error earlier or provide "dummy" implementation of some sort, instead it just crashes.

As far as jupyterlite is concerned, it looks like it simply doesn't allow for more than one message to be executed concurrently, and this restriction happens on the browser side, so there is nothing we can patch up on Python side to make this work. One has to complete cell execution event before any widget events can be delivered into Python. So if it's possible at all it would require a completely different strategy than what is currently used.

[drazellan]

Hello Thanks for answering.

" One has to complete cell execution event before any widget events can be delivered into Python. " this is bad news for what i was planning. I was unaware of this limitation.

[psychemedia]

Does the xeus-python kernel behave differently? https://github.com/jupyterlite/xeus

[Kirill888]

Does the xeus-python kernel behave differently? https://github.com/jupyterlite/xeus

@psychemedia Issue is in jupyterlite itself, messages get blocked inside JavaScript of jupyterlite, so before any kernel gets to see those.

[Kirill888]

2024: same issue remains with jupyterlite: processMsg is forced to process one message at a time

  async startNew(options: Kernels.IKernelOptions): Promise<Kernel.IModel> {
//...
    // create a synchronization mechanism to allow only one message
    // to be processed at a time
    const mutex = new Mutex();


    // hook a new client to a kernel
    const hook = (
      kernelId: string,
      clientId: string,
      socket: WebSocketClient,
    ): void => {
//...
      const processMsg = async (msg: KernelMessage.IMessage) => {
        await mutex.runExclusive(async () => {
          await kernel.ready;
          await kernel.handleMessage(msg);
        });

https://github.com/jupyterlite/jupyterlite/blob/eb0bcb543703213ea883dc1da948ce07494a3148/packages/kernel/src/kernels.ts#L44-L77