Remote servers: zmq-subprocess-error void-function

[brendan-r]

When trying to connect to kernels on remote notebook servers, I always get zmq-subprocess-error void-function. The server is in a docker container, on an EC2 instance.

emacs-jupyter works fantastic with a local kernel!

When running a notebook server via jupyter notebook --ip 0.0.0.0 --allow-root --no-browser:

• I can connect fine in the browser
• I can start a new remote kernels via emacs-jupyter
• I can list kernels via emacs-jupyter
• All tests pass in the zmq source dir (after downloading the test file)

However, I always get the same error when trying to connect to a kernel, for example if I press RET on a kernel listed in the buffer produced by jupyter-server-list-kernels. The full output with toggle-debug-on-error is under 'Error 1', below.

When killing the call to jupyter notebook ... and instead using jupyter kernel, and trying to connect via TRAMP syntax to a JSON file, I get Error 2.

Any ideas for me?

---

Package versions: zmq-20230214.36, jupyter-20230214.215 emacs-version: GNU Emacs 28.2 (build 1, x86_64-apple-darwin21.6.0, NS appkit-2113.60 Version 12.6.1 (Build 21G217)) of 2022-12-28. IIRC I didn't use native compilation.

Jupyter server info:

Selected Jupyter core packages...
IPython          : 7.34.0
ipykernel        : 6.12.1
ipywidgets       : 8.0.4
jupyter_client   : 7.4.9
jupyter_core     : 4.12.0
jupyter_server   : 1.23.5
jupyterlab       : 3.4.7
nbclient         : 0.7.2
nbconvert        : 7.2.9
nbformat         : 5.7.3
notebook         : 6.5.2
qtconsole        : 5.4.0
traitlets        : 5.9.0

Error 1:

Debugger entered--Lisp error: (zmq-subprocess-error void-function (nil))
  signal(zmq-subprocess-error (void-function (nil)))
  zmq--subprocess-filter(#<process zmq> "(error void-function (nil))")
  accept-process-output(nil 0.25)
  jupyter-ioloop-wait-until(#<jupyter-server-ioloop jupyter-server-ioloop-1fec9f67f430> start identity)
  #f(compiled-function (arg1 arg2 &rest rest) #<bytecode -0x1abb25ded472fdf9>)(#<jupyter-server-ioloop jupyter-server-ioloop-1fec9f67f430> #f(compiled-function (event) #<bytecode 0x1ada72949b509b07>))
  apply(#f(compiled-function (arg1 arg2 &rest rest) #<bytecode -0x1abb25ded472fdf9>) #<jupyter-server-ioloop jupyter-server-ioloop-1fec9f67f430> #f(compiled-function (event) #<bytecode 0x1ada72949b509b07>))
  jupyter-ioloop-start(#<jupyter-server-ioloop jupyter-server-ioloop-1fec9f67f430> #f(compiled-function (event) #<bytecode 0x1ada72949b509b07>))
  #f(compiled-function (comm) #<bytecode -0xdbdb7fd0264a18e>)(#<jupyter-server jupyter-server-1fec9f60346a>)
  apply(#f(compiled-function (comm) #<bytecode -0xdbdb7fd0264a18e>) #<jupyter-server jupyter-server-1fec9f60346a>)
  #f(compiled-function (&rest cnm-args) #<bytecode -0x84dc6af3e3cfea0>)()
  #f(compiled-function (cl--cnm comm) #<bytecode 0x10eec37be315cf46>)(#f(compiled-function (&rest cnm-args) #<bytecode -0x84dc6af3e3cfea0>) #<jupyter-server jupyter-server-1fec9f60346a>)
  apply(#f(compiled-function (cl--cnm comm) #<bytecode 0x10eec37be315cf46>) #f(compiled-function (&rest cnm-args) #<bytecode -0x84dc6af3e3cfea0>) #<jupyter-server jupyter-server-1fec9f60346a>)
  #f(compiled-function (&rest args) #<bytecode -0x159b7f0916a6a5fa>)(#<jupyter-server jupyter-server-1fec9f60346a>)
  apply(#f(compiled-function (&rest args) #<bytecode -0x159b7f0916a6a5fa>) #<jupyter-server jupyter-server-1fec9f60346a> nil)
  jupyter-comm-start(#<jupyter-server jupyter-server-1fec9f60346a>)
  #f(compiled-function (comm &rest ignore) "Register COMM to receive server events.\nIf SERVER receives events that have the same kernel ID as the\nkernel associated with COMM, then COMM's `jupyter-event-handler'\nwill receive those events." #<bytecode -0x18362b2c5cbdacda>)(#<jupyter-server-kernel-comm jupyter-server-kernel-comm-1fec9f644fca>)
  apply(#f(compiled-function (comm &rest ignore) "Register COMM to receive server events.\nIf SERVER receives events that have the same kernel ID as the\nkernel associated with COMM, then COMM's `jupyter-event-handler'\nwill receive those events." #<bytecode -0x18362b2c5cbdacda>) #<jupyter-server-kernel-comm jupyter-server-kernel-comm-1fec9f644fca> nil)
  jupyter-comm-start(#<jupyter-server-kernel-comm jupyter-server-kernel-comm-1fec9f644fca>)
  #f(compiled-function (manager) "Start a websocket connection to MANAGER's kernel.\nMANAGER's COMM slot will be set to the `jupyter-comm-layer'\nreceiving events on the websocket when this method returns." #<bytecode -0x134ffb6f8a935e4f>)(#<jupyter-server-kernel-manager jupyter-server-kernel-manager-1feca6fe755c>)
  apply(#f(compiled-function (manager) "Start a websocket connection to MANAGER's kernel.\nMANAGER's COMM slot will be set to the `jupyter-comm-layer'\nreceiving events on the websocket when this method returns." #<bytecode -0x134ffb6f8a935e4f>) #<jupyter-server-kernel-manager jupyter-server-kernel-manager-1feca6fe755c> nil)
  jupyter-comm-start(#<jupyter-server-kernel-manager jupyter-server-kernel-manager-1feca6fe755c>)
  #f(compiled-function (manager class &rest slots) #<bytecode -0xf8b54f488f84712>)(#f(compiled-function (&rest cnm-args) #<bytecode 0x6396485e6cbaefb>) #<jupyter-server-kernel-manager jupyter-server-kernel-manager-1feca6fe755c> jupyter-repl-client)
  apply(#f(compiled-function (manager class &rest slots) #<bytecode -0xf8b54f488f84712>) #f(compiled-function (&rest cnm-args) #<bytecode 0x6396485e6cbaefb>) (#<jupyter-server-kernel-manager jupyter-server-kernel-manager-1feca6fe755c> jupyter-repl-client))
  #f(compiled-function (&rest args) #<bytecode 0x1ed8f48b6663c234>)(#<jupyter-server-kernel-manager jupyter-server-kernel-manager-1feca6fe755c> jupyter-repl-client)
  apply(#f(compiled-function (&rest args) #<bytecode 0x1ed8f48b6663c234>) (#<jupyter-server-kernel-manager jupyter-server-kernel-manager-1feca6fe755c> jupyter-repl-client))
  #f(compiled-function (&rest args) #<bytecode -0x158969df9a4882aa>)(#<jupyter-server-kernel-manager jupyter-server-kernel-manager-1feca6fe755c> jupyter-repl-client)
  apply(#f(compiled-function (&rest args) #<bytecode -0x158969df9a4882aa>) #<jupyter-server-kernel-manager jupyter-server-kernel-manager-1feca6fe755c> jupyter-repl-client)
  jupyter-make-client(#<jupyter-server-kernel-manager jupyter-server-kernel-manager-1feca6fe755c> jupyter-repl-client)
  jupyter-connect-server-repl(#<jupyter-server jupyter-server-1fec9f60346a> "eab186f1-ae37-4a06-a21b-ec97c191ba73")
  jupyter-server-kernel-list-new-repl()
  funcall-interactively(jupyter-server-kernel-list-new-repl)
  call-interactively(jupyter-server-kernel-list-new-repl nil nil)
  command-execute(jupyter-server-kernel-list-new-repl)

Error 2:

Debugger entered--Lisp error: (error "Kernel did not respond to kernel-info request")
  signal(error ("Kernel did not respond to kernel-info request"))
  error("Kernel did not respond to kernel-info request")
  jupyter-kernel-info(#<jupyter-org-client jupyter-org-client-1fec9f641c18>)
  #f(compiled-function (client &optional repl-name associate-buffer display) #<bytecode 0x3897f914c3a6735>)(#<jupyter-org-client jupyter-org-client-1fec9f641c18> nil nil nil)
  apply(#f(compiled-function (client &optional repl-name associate-buffer display) #<bytecode 0x3897f914c3a6735>) (#<jupyter-org-client jupyter-org-client-1fec9f641c18> nil nil nil))
  #f(compiled-function (&rest args) #<bytecode 0x15d718fd21442c55>)(#<jupyter-org-client jupyter-org-client-1fec9f641c18> nil nil nil)
  apply(#f(compiled-function (&rest args) #<bytecode 0x15d718fd21442c55>) #<jupyter-org-client jupyter-org-client-1fec9f641c18> (nil nil nil))
  jupyter-bootstrap-repl(#<jupyter-org-client jupyter-org-client-1fec9f641c18> nil nil nil)
  jupyter-connect-repl("/ssh:REDACTED@host|docker:root@015d03a08ee6:/root/..." nil nil jupyter-org-client)
  #f(compiled-function (session kernel) "Initiate a client connected to a remote kernel process." #<bytecode -0x8aa945e55ec9b70>)(#s(org-babel-jupyter-remote-session :name "/ssh:REDACTED@host|docker:root@015d03a08ee6:/root/..." :connect-repl-p t) "python3")
  apply(#f(compiled-function (session kernel) "Initiate a client connected to a remote kernel process." #<bytecode -0x8aa945e55ec9b70>) (#s(org-babel-jupyter-remote-session :name "/ssh:redacted@host|docker:root@015d03a08ee6:/root/..." :connect-repl-p t) "python3"))
  #f(compiled-function (&rest args) #<bytecode -0x1355e2a81368f7ab>)(#s(org-babel-jupyter-remote-session :name "/ssh:REDACTED@host|docker:root@015d03a08ee6:/root/..." :connect-repl-p t) "python3")
  apply(#f(compiled-function (&rest args) #<bytecode -0x1355e2a81368f7ab>) (#s(org-babel-jupyter-remote-session :name "/ssh:redacted@host|docker:root@015d03a08ee6:/root/..." :connect-repl-p t) "python3"))
  #f(compiled-function (&rest cnm-args) #<bytecode 0x1f010df4c37e68>)()
  #f(compiled-function (cl--cnm session kernel) "Rename the returned client's REPL buffer to include SESSION's name.\nAlso set `jupyter-include-other-output' to nil for the session so\nthat output produced by other clients do not get handled by the\nclient." #<bytecode 0x2a0adbe19a0cb42>)(#f(compiled-function (&rest cnm-args) #<bytecode 0x1f010df4c37e68>) #s(org-babel-jupyter-remote-session :name "/ssh:redacted@host|docker:root@015d03a08ee6:/root/..." :connect-repl-p t) "python3")
  apply(#f(compiled-function (cl--cnm session kernel) "Rename the returned client's REPL buffer to include SESSION's name.\nAlso set `jupyter-include-other-output' to nil for the session so\nthat output produced by other clients do not get handled by the\nclient." #<bytecode 0x2a0adbe19a0cb42>) #f(compiled-function (&rest cnm-args) #<bytecode 0x1f010df4c37e68>) (#s(org-babel-jupyter-remote-session :name "/ssh:redacted@host|docker:root@015d03a08ee6:/root/..." :connect-repl-p t) "python3"))
  #f(compiled-function (&rest args) #<bytecode 0x573fe3dfc033526>)(#s(org-babel-jupyter-remote-session :name "/ssh:redacted@host|docker:root@015d03a08ee6:/root/..." :connect-repl-p t) "python3")
  apply(#f(compiled-function (&rest args) #<bytecode 0x573fe3dfc033526>) #s(org-babel-jupyter-remote-session :name "/ssh:redacted@host|docker:root@015d03a08ee6:/root/..." :connect-repl-p t) "python3")
  org-babel-jupyter-initiate-client(#s(org-babel-jupyter-remote-session :name "/ssh:redacted@host|docker:root@015d03a08ee6:/root/..." :connect-repl-p t) "python3")
  org-babel-jupyter-initiate-session-by-key("/ssh:redacted@host|docker:root@015d03a08ee6:/root/..." ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "yes") (:session . "/ssh:redacted@host|docker:root@015d03a08ee6:/root/...")))
  org-babel-jupyter-initiate-session("/ssh:redacted@host|docker:root@015d03a08ee6:/root/..." ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "yes") (:session . "/ssh:redacted@host|docker:root@015d03a08ee6:/root/...")))
  org-babel-execute:jupyter-python("4 + 4" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "yes") (:session . "/ssh:redacted@host|docker:root@015d03a08ee6:/root/...")))
  org-babel-execute-src-block(nil ("jupyter-python" "4 + 4" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:session . "/ssh:redacted@host|docker:root@015d03a08ee6:/root/...") (:async . "yes") (:kernel . "python3") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 74 "(ref:%s)"))
  org-ctrl-c-ctrl-c(nil)
  funcall-interactively(org-ctrl-c-ctrl-c nil)
  call-interactively(org-ctrl-c-ctrl-c nil nil)
  command-execute(org-ctrl-c-ctrl-c)

[nnicandro]

Could you try out the next branch. With that branch, when communicating with a notebook kernel there is no intermediate jupyter-ioloop so Error 1 shouldn't happen anymore. I don't know what results in Error 2, but it might be fixed on the next branch as well.

[baruch11]

Could you try out the next branch. With that branch, when communicating with a notebook kernel there is no intermediate jupyter-ioloop so Error 1 shouldn't happen anymore. I don't know what results in Error 2, but it might be fixed on the next branch as well.

It worked for me, thanks. (I have the same error (Error1) on branch master, and it works on 'next' branch)