Generic way for kernels to expose comms

[martinRenou]

Currently, ipywidgets is relying on ipykernel for making use of the Comms.

It would be nice to have a more generic way for libraries to access the Comm implementation of the currently running kernel. Right now, xeus-python monkey-patches ipykernel in order to inject its own Comm class.

One idea would be to use dynamic instrumentation with importlib in the kernel implementation in order to inject a fake module, let's say we name it current_kernel. Then libraries like ipywidgets would import the Comm implementation doing:

from current_kernel import Comm

Another idea would be to use entry points, kernel would expose their Comm implementation and other useful things through entry points. But we would need to figure out a way for libraries to know which kernel is currently running.

Those two solutions are very specific to Python though. It would be nicer to find a solution that could be expressed in the Jupyter kernel protocol and that could be implemented in any language (maybe a file-based solution?).

cc. @jasongrout @SylvainCorlay @carreau

[martinRenou]

Another idea. Currently, everyone expects display to be exposed to the global scope. Maybe we should simply expect the kernel to expose a well-defined set of tools to the global scope.

[Carreau]

I think that make sens; Do you want to register comm on PyPI ?

[martinRenou]

Do you want to register comm on PyPI ?

What do you mean?

[Carreau]

from current_kernel import Comm

You need current_kernel to not be a taken python package name, and it's too long, and/or make sure that the package is cooperating with current kernel. I suggest

from comm import Comm

to make sure comm is not, or will not be another package, I registered it https://pypi.org/project/comm/

So now we are clear to use from comm import ..., and we can add login in the core comm package to re-export things from ipykernel so that new packages using from comm import ... will work on older version of IPython/ipykernel.

[martinRenou]

Oh I see, it makes sense yes.

Maybe something more general than Comm might be better though, because we might want to do the same for some other useful features accessible using the current get_ipython, and put all those features in a single fake module.

[jasongrout]

So now we are clear to use from comm import ...,

I like the idea of having a comm package, and kernels can depend on it and register themselves as the current comm implementation. So I imagine a kernel does something like this at startup, maybe with a fallback to use ipykernel if it is importable.

from comm import register_comm_provider
register_comm_provider(my_comm_provider)

Then packages can do something like:

from comm import Comm
Comm(...)

and not have to depend on ipykernel.

[jasongrout]

I like the idea of having a comm package, and kernels can depend on it and register themselves as the current comm implementation.

If I understand things correctly, this is roughly the pattern used in Python's asyncio package, where various packages can register event loop implementations, but people writing asyncio code don't have to know or depend on a specific implementation.