ipyparallel icon indicating copy to clipboard operation
ipyparallel copied to clipboard

IPykernel 7 compatibility.

Open Carreau opened this issue 1 year ago • 5 comments

It looks like ipyparallel is not compatible with ipykernel 7,

Unfortunately I've not been following the development, and the previous maintainer stepped down a few month ago, and I'm afraid of the 6.x and 7.x branch diverging.

I would love to get some help get the downstream testing passing on all projects that depends on ipykernel.

cf https://github.com/ipython/ipykernel/issues/1222

Carreau avatar Oct 15 '24 09:10 Carreau

Thanks, I think I can work on this this week.

minrk avatar Oct 15 '24 18:10 minrk

I think I've tracked it down to the incomplete assumption of async sockets in the updates to IOThread. Looking into more detail, but I think the fix belongs in ipykernel because there's a mixture of sync and async use of sockets, relying on the assumption that async zmq sends are eager and don't need to be awaited.

minrk avatar Oct 21 '24 10:10 minrk

I do trust you, I haven't followed what has append in ipykernel, and I'm mostly trying to not have everything fall appart.

Carreau avatar Oct 21 '24 11:10 Carreau

Looks like it's a little more complicated than that in that ipyparallel relied on support for shell_streams as a list of sockets on which to handle shell messages, and either the anyio or subshell work removed the ability to handle multiple sockets. That will probably need at least some work on this side, possibly both.

minrk avatar Oct 21 '24 12:10 minrk

Thinking out loud about how to address the single socket issue

Two choices are:

  1. restore support for a list of shell sockets in ipykernel. This makes less sense for ipykernel now that control has its own special handling, and is prone to feature removal, as has been done in ipykernel main
  2. let ipykernel keep a single socket, and implement a simple proxy in ipyparallel to funnel down to the single socket. This will perhaps be the easiest to maintain because ipykernel doesn't need to support a feature it doesn't use, at the expense of adding an inproc socket hop on top of the one already added for subshells.

I'm leaning toward the latter, which I think wouldn't need any change from ipykernel

minrk avatar Oct 21 '24 13:10 minrk

I was wrong about that, ipyparallel uses one socket connected to multiple URLs, so setting shell_socket works okay

minrk avatar Oct 22 '24 12:10 minrk