jupyter
jupyter copied to clipboard
emacs-jupyter
Cannot execute kernels due to ZMQ errors
I didn't have a better title for the issue, my apologies for that. I am encountering a problem when attempting to run any kernel (Julia, Sage, Python tested) for the first time.
When I execute a kernel for the first time, I get the following error (debug on error is toggled on):
Debugger entered--Lisp error: (error "Need a valid session and a kernel to form a key")
error("Need a valid session and a kernel to form a key")
org-babel-jupyter-session-key(((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session . "pyth")))
org-babel-jupyter-initiate-session-by-key("pyth" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session . "pyth")))
org-babel-jupyter-initiate-session("pyth" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session . "pyth")))
org-babel-execute:jupyter-python("print(\"yy\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session . "pyth")))
#<subr org-babel-execute-src-block>(nil ("jupyter-python" "print(\"yy\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:session . "pyth") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 661 "(ref:%s)") nil nil)
apply(#<subr org-babel-execute-src-block> (nil ("jupyter-python" "print(\"yy\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:session . "pyth") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 661 "(ref:%s)")))
org-babel-execute-src-block(nil ("jupyter-python" "print(\"yy\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:session . "pyth") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 661 "(ref:%s)"))
org-ctrl-c-ctrl-c(nil)
funcall-interactively(org-ctrl-c-ctrl-c nil)
command-execute(org-ctrl-c-ctrl-c)
If I try to execute the kernel a second time, I get a different error:
Debugger entered--Lisp error: (zmq-subprocess-error error ("Invalid signature (c3ac79e384577619b9e81b0afaacf08..."))
signal(zmq-subprocess-error (error ("Invalid signature (c3ac79e384577619b9e81b0afaacf08...")))
zmq--subprocess-filter(#<process zmq> "(error error (\"Invalid signature (c3ac79e384577619...")
accept-process-output(nil 3.0)
jupyter-wait-until-idle(#s(jupyter-org-request :id "0fa2f9bc-70cc-40e5-a768-8cb4dd3df56c" :type "kernel_info_request" :content nil :client #<jupyter-org-client jupyter-org-client-22c2b02> :time (25735 4384 270090 396000) :idle-p nil :last-message nil :messages nil :message-publisher #f(compiled-function (pub-value) #<bytecode -0x1679e420a9ccc046>) :inhibited-handlers t :result-type value :block-params ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "no") (:session . "pyth")) :file nil :results nil :silent-p nil :id-cleared-p nil :inline-block-p nil :marker #<marker at 661 in try_jupyter.org> :async-p nil) 30)
#f(compiled-function (req) #<bytecode -0x138a9485a510db2e>)(#s(jupyter-org-request :id "0fa2f9bc-70cc-40e5-a768-8cb4dd3df56c" :type "kernel_info_request" :content nil :client #<jupyter-org-client jupyter-org-client-22c2b02> :time (25735 4384 270090 396000) :idle-p nil :last-message nil :messages nil :message-publisher #f(compiled-function (pub-value) #<bytecode -0x1679e420a9ccc046>) :inhibited-handlers t :result-type value :block-params ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "no") (:session . "pyth")) :file nil :results nil :silent-p nil :id-cleared-p nil :inline-block-p nil :marker #<marker at 661 in try_jupyter.org> :async-p nil))
#f(compiled-function (state) #<bytecode -0x1d90e9b38506d041>)(#<jupyter-org-client jupyter-org-client-22c2b02>)
#f(compiled-function (state) #<bytecode -0x195f480951d402bc>)(#<jupyter-org-client jupyter-org-client-22c2b02>)
#f(compiled-function (state) #<bytecode 0x11c097ad5a23b7bd>)(#<jupyter-org-client jupyter-org-client-22c2b02>)
#f(compiled-function (state) #<bytecode -0x16a67099f1a6589e>)(#<jupyter-org-client jupyter-org-client-22c2b02>)
jupyter-run-with-state(#<jupyter-org-client jupyter-org-client-22c2b02> #f(compiled-function (state) #<bytecode -0x16a67099f1a6589e>))
jupyter-kernel-info(#<jupyter-org-client jupyter-org-client-22c2b02>)
#f(compiled-function (kernel &optional client-class) #<bytecode 0x3ff9c360bbc2c5>)(#<jupyter-kernel python3> jupyter-org-client)
apply(#f(compiled-function (kernel &optional client-class) #<bytecode 0x3ff9c360bbc2c5>) #<jupyter-kernel python3> jupyter-org-client)
jupyter-client(#<jupyter-kernel python3> jupyter-org-client)
jupyter-run-repl("python3" nil nil jupyter-org-client)
#f(compiled-function (session kernel) "Launch SESSION's KERNEL, return a `jupyter-org-client' connected to it.\nSESSION is the :session header argument of a source block and\nKERNEL is the name of the kernel to launch." #<bytecode -0xdbacf1d8f03860c>)(#s(org-babel-jupyter-session :name "pyth") "python3")
apply(#f(compiled-function (session kernel) "Launch SESSION's KERNEL, return a `jupyter-org-client' connected to it.\nSESSION is the :session header argument of a source block and\nKERNEL is the name of the kernel to launch." #<bytecode -0xdbacf1d8f03860c>) (#s(org-babel-jupyter-session :name "pyth") "python3"))
#f(compiled-function (&rest args) #<bytecode -0x7ebea9bd63424d5>)()
#f(compiled-function (cl--cnm session kernel) #<bytecode 0xa9e648033d11ff5>)(#f(compiled-function (&rest args) #<bytecode -0x7ebea9bd63424d5>) #s(org-babel-jupyter-session :name "pyth") "python3")
apply(#f(compiled-function (cl--cnm session kernel) #<bytecode 0xa9e648033d11ff5>) #f(compiled-function (&rest args) #<bytecode -0x7ebea9bd63424d5>) (#s(org-babel-jupyter-session :name "pyth") "python3"))
#f(compiled-function (session kernel) "Rename the returned client's REPL buffer to include SESSION's name.\nAlso set `jupyter-include-other-output' to nil for the session so\nthat output produced by other clients do not get handled by the\nclient." #<bytecode 0x1005fc003ec275c4>)(#s(org-babel-jupyter-session :name "pyth") "python3")
apply(#f(compiled-function (session kernel) "Rename the returned client's REPL buffer to include SESSION's name.\nAlso set `jupyter-include-other-output' to nil for the session so\nthat output produced by other clients do not get handled by the\nclient." #<bytecode 0x1005fc003ec275c4>) #s(org-babel-jupyter-session :name "pyth") "python3")
org-babel-jupyter-initiate-client(#s(org-babel-jupyter-session :name "pyth") "python3")
org-babel-jupyter-initiate-session-by-key("pyth" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "no") (:session . "pyth")))
org-babel-jupyter-initiate-session("pyth" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "no") (:session . "pyth")))
org-babel-execute:jupyter-python("print(\"yy\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "no") (:session . "pyth")))
org-babel-execute-src-block(nil ("jupyter-python" "print(\"yy\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:session . "pyth") (:async . "no") (:kernel . "python3") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 661 "(ref:%s)"))
org-ctrl-c-ctrl-c(nil)
funcall-interactively(org-ctrl-c-ctrl-c nil)
command-execute(org-ctrl-c-ctrl-c)
How can this issue be solved? I am testing with very simple code:
#+begin_src jupyter-sage :session math
print("A")
#+end_src
#+begin_src jupyter-julia :session cesar
println("B")
#+end_src
#+begin_src jupyter-python :session nissa
print("C")
#+end_src
Additional context:
- NixOS unstable flake
- GNU Emacs 30.0.50
I have done some debugging on this. Turns out that setting (setq jupyter-use-zmq nil) can be used as a workaround. The first evaluation of the kernels gives a bunch of CL errors:
Executing Jupyter-Sage code block at position 481...
Launching notebook process on port 8888...done
Launching sagemath kernel...done
Requesting kernel info...
Timeout before idle: #s(jupyter-org-request "dc9da6bc-e960-4cc3-8d46-85a228dd82ba" "kernel_info_request" nil #s(jupyter-org-client jupyter--clients (#<finalizer>) "idle" 1 nil #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) (#[257 "$\0A@#A@A\"\"\0B\"" [#[257 " \0!:\0\0!:\0:}\0:{\0:y\0?w\0!>\\\0D\"H\"&\"\0!?\0 !" [(#[0 "!&" ["d0a9c497-91db-462c-8d1b-72a900945148" #s(jupyter-server "http://localhost:8888" "ws://localhost:8888" t ...) #[514 "!
>\0D\"H>5\0!
>(\0D\"H!B!\"!
>J\0D\"HD!\"" [#[257 "$\0A@#A@A\"\"\0B\"" [jupyter-content ... #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 8 subscribe 16)) jupyter-pseudo-bind-content memql error "Unhandled publisher content: %s"] 8 "
(fn PUB-VALUE)"] #3 cl-struct-websocket-frame-tags type-of signal wrong-type-argument websocket-frame 1 ... jupyter-read-plist-from-string ...] 11 "
(fn WS FRAME)"] jupyter-api-kernel-websocket :custom-header-alist jupyter-api-auth-headers :on-message] 7]) (nil) (#s(websocket connecting (:id "d0a9c497-91db-462c-8d1b-72a900945148" :session #s(jupyter-session nil "3b8cfdc8-3aca-4588-9418-3efaf21d4e08" nil)) identity #[514 "!
>\0D\"H>5\0!
>(\0D\"H!B!\"!
>J\0D\"HD!\"" [#[257 "$\0A@#A@A\"\"\0B\"" [jupyter-content ... #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 8 subscribe 16)) jupyter-pseudo-bind-content memql error "Unhandled publisher content: %s"] 8 "
(fn PUB-VALUE)"] #3 cl-struct-websocket-frame-tags type-of signal wrong-type-argument websocket-frame 1 ... jupyter-read-plist-from-string ...] 11 "
(fn WS FRAME)"] identity websocket-default-error-handler nil nil nil ...)) cl-struct-websocket-tags error "Kernel shutdown!" #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (message 23 send 29)) jupyter-content websocket-send-text type-of ...] 24 "
(fn EVENT)"] (subscribers #[257 "$\0A@#A@A\"\"\0B\"" [#[257 "!
>\0D\"H-\0\"#\0!\"-\0C!\"?\0!\"\0!
>T\0D\"I!
>h\0D\"HCI\"\0!\"\0!\"\0\"\0!\"\0\"\0!\"\0!
>\0D\"IBB!" ["dc9da6bc-e960-4cc3-8d46-85a228dd82ba" #0 cl-struct-jupyter-request-tags type-of signal wrong-type-argument jupyter-request 6 plist-get :msg_type ...] 7 "
(fn MSG)"] (subscribers #[257 ":L\0G\0:&\0 \0!\"\":B\0=\0\"\"\"\"\"" [#[257 "\"#" [#1 plist-get :channel jupyter-handle-message] 6 "
(fn MSG)"] #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 13 subscribe 43)) error "Unhandled subscriber content: %s" signal jupyter-subscribed-subscriber nil] 8 "
(fn SUB-CONTENT)"]) #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 8 subscribe 16)) jupyter-pseudo-bind-content memql error "Unhandled publisher content: %s"] 8 "
(fn PUB-VALUE)"]) #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 8 subscribe 16)) jupyter-pseudo-bind-content memql error "Unhandled publisher content: %s"] 8 "
(fn PUB-VALUE)"] #[257 ":L\0G\0:&\0 \0!\"\":B\0=\0\"\"\"\"\"" [#[257 "\0!!!!:A\0=?\0:=\0?;\0!" [(nil) (#s(websocket connecting (:id "d0a9c497-91db-462c-8d1b-72a900945148" :session #s(jupyter-session nil "3b8cfdc8-3aca-4588-9418-3efaf21d4e08" nil)) identity #[514 "!
>\0D\"H>5\0!
>(\0D\"H!B!\"!
>J\0D\"HD!\"" [#[257 "$\0A@#A@A\"\"\0B\"" [jupyter-content ... #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 8 subscribe 16)) jupyter-pseudo-bind-content memql error "Unhandled publisher content: %s"] 8 "
(fn PUB-VALUE)"] #[257 "$\0A@#A@A\"\"\0B\"" [#[257 " \0!:\0\0!:\0:}\0:{\0:y\0?w\0!>\\\0D\"H\"&\"\0!?\0 !" [... ... #7 cl-struct-websocket-tags error "Kernel shutdown!" #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (message 23 send 29)) jupyter-content websocket-send-text type-of ...] 24 "
(fn EVENT)"] ... #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 8 subscribe 16)) jupyter-pseudo-bind-content memql error "Unhandled publisher content: %s"] 8 "
(fn PUB-VALUE)"] cl-struct-websocket-frame-tags type-of signal wrong-type-argument websocket-frame 1 ... jupyter-read-plist-from-string ...] 11 "
(fn WS FRAME)"] identity websocket-default-error-handler nil nil nil ...)) #s(jupyter-server-kernel #s(jupyter-kernelspec "sagemath" (:argv ["/home/loren/.nix-profile/bin/sage" "--python" "-m" "sage.repl.ipython_kernel" "-f" "{connection_file}"] :env nil :display_name "SageMath 10.0" :language "sage" :interrupt_mode "signal" ...) nil) #s(jupyter-session nil "fd6c2fc2-954a-4b8f-a8e3-b9f2a08e856f" nil) #s(jupyter-server "http://localhost:8888" "ws://localhost:8888" t (:default "python3" :kernelspecs ...)) "d0a9c497-91db-462c-8d1b-72a900945148") #s(hash-table size 3 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (interrupt 6 shutdown 10 restart 23)) jupyter-interrupt jupyter-shutdown websocket-close t jupyter-restart action] 7 "
(fn ACTION)"] #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 13 subscribe 43)) error "Unhandled subscriber content: %s" signal jupyter-subscribed-subscriber nil] 8 "
(fn SUB-CONTENT)"]) #<buffer *jupyter-kernel-client*> nil ...) (25742 64481 76558 673000) nil nil nil #[257 "$\0A@#A@A\"\"\0B\"" [#[257 "!
>\0D\"H-\0\"#\0!\"-\0C!\"?\0!\"\0!
>T\0D\"I!
>h\0D\"HCI\"\0!\"\0!\"\0\"\0!\"\0\"\0!\"\0!
>\0D\"IBB!" ["dc9da6bc-e960-4cc3-8d46-85a228dd82ba" #0 cl-struct-jupyter-request-tags type-of signal wrong-type-argument jupyter-request 6 plist-get :msg_type ...] 7 "
(fn MSG)"] (subscribers #[257 ":L\0G\0:&\0 \0!\"\":B\0=\0\"\"\"\"\"" [#[257 "\"#" [#s(jupyter-org-client jupyter--clients (#<finalizer>) "idle" 1 nil #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data ()) (#[257 "$\0A@#A@A\"\"\0B\"" [#[257 " \0!:\0\0!:\0:}\0:{\0:y\0?w\0!>\\\0D\"H\"&\"\0!?\0 !" [... ... ... cl-struct-websocket-tags error "Kernel shutdown!" #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (message 23 send 29)) jupyter-content websocket-send-text type-of ...] 24 "
(fn EVENT)"] ... #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 8 subscribe 16)) jupyter-pseudo-bind-content memql error "Unhandled publisher content: %s"] 8 "
(fn PUB-VALUE)"] #[257 ":L\0G\0:&\0 \0!\"\":B\0=\0\"\"\"\"\"" [#[257 "\0!!!!:A\0=?\0:=\0?;\0!" [... ... #s(jupyter-server-kernel #s(jupyter-kernelspec "sagemath" ... nil) #s(jupyter-session nil "fd6c2fc2-954a-4b8f-a8e3-b9f2a08e856f" nil) #s(jupyter-server "http://localhost:8888" "ws://localhost:8888" t ...) "d0a9c497-91db-462c-8d1b-72a900945148") #s(hash-table size 3 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (interrupt 6 shutdown 10 restart 23)) jupyter-interrupt jupyter-shutdown websocket-close t jupyter-restart action] 7 "
(fn ACTION)"] #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 13 subscribe 43)) error "Unhandled subscriber content: %s" signal jupyter-subscribed-subscriber nil] 8 "
(fn SUB-CONTENT)"]) #<buffer *jupyter-kernel-client*> nil ...) plist-get :channel jupyter-handle-message] 6 "
(fn MSG)"] #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 13 subscribe 43)) error "Unhandled subscriber content: %s" signal jupyter-subscribed-subscriber nil] 8 "
(fn SUB-CONTENT)"]) #s(hash-table size 2 test eq rehash-size 1.5 rehash-threshold 0.8125 purecopy t data (content 8 subscribe 16)) jupyter-pseudo-bind-content memql error "Unhandled publisher content: %s"] 8 "
(fn PUB-VALUE)"] ...)
Then the second evaluation works just fine. I also played with individual commits, and somehow after 11ace8c805e4f866005a77891b29c4fe036c57c8 I do not get syntax highlighting on the org-mode code blocks, I don't know if this is meant to be like that.
I also get this... when using (setq jupyter-use-zmq nil) first block execution always fails. Needs to wait a bit and retry and then it works...
Latest emacs-jupyter requires to set a :kernel <kernel-name> argument which will be the name of your jupyter kernel.
Seeing the same behavior as OP with NixOS stable (23.05) and Emacs 29.1.
@Panadestein curious which combination of libraries you use, maybe we get more clues that way. I use zeromq, python311Packages.pyzmq (FWIW) and zmq.el from emacsWithPackages - all from nixpkgs (the official NixOS package repository).
@fleimgruber sorry for the late reply. I am using the nixpkgs-unstable flake, so basically the latest packages available. I use straight.el for managing Emacs packages. Furthermore, I use the zeromq Emacs package, and I had to pin emacs-jupyter to the commit 455166712e606c9c6a8de763ea0a77548cadcef2. If I use any older commit, I start experiencing the issues I mentioned above.
Just to report that I'm observing the same issue: I cannot execute org blocks because I get the error "Invalid Signature".
I also randomly started getting weird glibc zmq errors. I was able to fix this by using older versions of the zmq and jupyter packages (which I configured using elpaca).
For jupyter, I used https://github.com/emacs-jupyter/jupyter/commit/455166712e606c9c6a8de763ea0a77548cadcef2, as mentioned by @Panadestein.
For zmq, I used https://github.com/nnicandro/emacs-zmq/commit/af5299d80715b1083a18145e9c84ef9563020676.
For jupyter, I used 4551667, as mentioned by @Panadestein.
For zmq, I used nnicandro/emacs-zmq@af5299d.
I can confirm that this works but only for emacs 28. For emacs 29, the issue still persists:
zmq--subprocess-filter: Error in ZMQ subprocess: error, ("Invalid signature (b2a362374ce1f32fb13e587c574fc68dcb6083f41809d450e71bee247aeb414c) for parts (\"{\\\"msg_id\\\": \\\"4a3353b1-406a5f9bdd724b89fa615f2d_1444615_1\\\", \\\"msg_type\\\": \\\"stream\\\", \\\"username\\\": \\\"bla\\\", \\\"session\\\": \\\"4a3353b1-406a5f9bdd724b89fa615f2d\\\", \\\"date\\\": \\\"2023-09-12T02:09:48.216049Z\\\", \\\"version\\\": \\\"5.3\\\"}\" \"{}\" \"{}\" \"{\\\"name\\\": \\\"stderr\\\", \\\"text\\\": \\\"--- Logging error ---\\\\nTraceback (most recent call last):\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/site-packages/ipykernel/kernelbase.py\\\\\\\", line 365, in dispatch_shell\\\\n msg = self.session.deserialize(msg, content=True, copy=False)\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/site-packages/jupyter_client/session.py\\\\\\\", line 1070, in deserialize\\\\n raise ValueError(msg)\\\\nValueError: Invalid Signature: b'710e70242cf943ee23e870e409e47a54688926ff75e2622653f8a612f83c70f3'\\\\n\\\\nDuring handling of the above exception, another exception occurred:\\\\n\\\\nTraceback (most recent call last):\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/logging/__init__.py\\\\\\\", line 1087, in emit\\\\n self.flush()\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/logging/__init__.py\\\\\\\", line 1067, in flush\\\\n self.stream.flush()\\\\nOSError: [Errno 5] Input/output error\\\\nCall stack:\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/runpy.py\\\\\\\", line 197, in _run_module_as_main\\\\n return _run_code(code, main_globals, None,\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/runpy.py\\\\\\\", line 87, in _run_code\\\\n exec(code, run_globals)\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/site-packages/ipykernel_launcher.py\\\\\\\", line 17, in <module>\\\\n app.launch_new_instance()\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/site-packages/traitlets/config/application.py\\\\\\\", line 992, in launch_instance\\\\n app.start()\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/site-packages/ipykernel/kernelapp.py\\\\\\\", line 711, in start\\\\n self.io_loop.start()\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/site-packages/tornado/platform/asyncio.py\\\\\\\", line 215, in start\\\\n self.asyncio_loop.run_forever()\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/asyncio/base_events.py\\\\\\\", line 601, in run_forever\\\\n self._run_once()\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/asyncio/base_events.py\\\\\\\", line 1905, in _run_once\\\\n handle._run()\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/asyncio/events.py\\\\\\\", line 80, in _run\\\\n self._context.run(self._callback, *self._args)\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/site-packages/ipykernel/kernelbase.py\\\\\\\", line 510, in dispatch_queue\\\\n await self.process_one()\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/site-packages/ipykernel/kernelbase.py\\\\\\\", line 499, in process_one\\\\n await dispatch(*args)\\\\n File \\\\\\\"/home/bla/miniconda3/envs/tmp/lib/python3.9/site-packages/ipykernel/kernelbase.py\\\\\\\", line 367, in dispatch_shell\\\\n self.log.error(\\\\\\\"Invalid Message\\\\\\\", exc_info=True)\\\\nMessage: 'Invalid Message'\\\\nArguments: ()\\\\n\\\"}\")")
@ugurbolat, I'm using Emacs 29 and things are working for me.
I downgraded emacs-jupyter to the commit "053a78da252b19cf59cefe6b83f9c4531a38d6b2". If you are using doom emacs by any chance, this is the line that I have in my packages.el file:
(package! jupyter :pin "053a78da252b19cf59cefe6b83f9c4531a38d6b2")
@UndeadKernel Yes, I am in doom (but will test in vanilla emacs too)
(package! emacs-zmq
:recipe (:host github
:repo "nnicandro/emacs-zmq")
:pin "af5299d80715b1083a18145e9c84ef9563020676")
(package! jupyter
:recipe (:host github
:repo "emacs-jupyter/jupyter")
:pin "053a78da252b19cf59cefe6b83f9c4531a38d6b2")
Now I have different error thrown:
Warning: Unknown type: zmq-socket
error in process filter: zmq--subprocess-read-output: Invalid read syntax: "#", 1, 3
error in process filter: Invalid read syntax: "#", 1, 3
Warning: Unknown type: zmq-socket
Assertion failed: (process-live-p process)finalizer failed: (cl-no-primary-method jupyter-kill-kernel #s(jupyter-kernel-process-manager (#<finalizer used>) jupyter--kernel-managers #s(jupyter-spec-kernel (#<finalizer> #<finalizer used>) ("python3" "/home/bolatu/miniconda3/envs/tmp/share/jupyter/kernels/python3" :argv ["/home/bolatu/miniconda3/envs/tmp/bin/python" "-m" "ipykernel_launcher" "-f" "{connection_file}"] :env nil :display_name "Python 3 (ipykernel)" :language "python" :interrupt_mode "signal" :metadata (:debugger t)) #s(jupyter-session (:kernel_name "python3" :transport "tcp" :ip "127.0.0.1" :signature_scheme "hmac-sha256" :key "d08db55a-1d17-4afd-9ef1-9855b97d0c7b" :hb_port 42367 :stdin_port 45679 :control_port 34039 :shell_port 40597 :iopub_port 44011) "ff09638e-3bb9-46c3-8a95-db3b3c3c96d0" "d08db55a-1d17-4afd-9ef1-9855b97d0c7b") #<process jupyter-kernel-python3>) #s(jupyter-zmq-channel :control #s(jupyter-session (:kernel_name "python3" :transport "tcp" :ip "127.0.0.1" :signature_scheme "hmac-sha256" :key "d08db55a-1d17-4afd-9ef1-9855b97d0c7b" :hb_port 42367 :stdin_port 45679 :control_port 34039 :shell_port 40597 :iopub_port 44011) "ff09638e-3bb9-46c3-8a95-db3b3c3c96d0" "d08db55a-1d17-4afd-9ef1-9855b97d0c7b") "tcp://127.0.0.1:34039" #<user-ptr ptr=0x56014ba44190 finalizer=0x7f5de1cb6e50>)))
@ugurbolat, I didn't have to pin emancs-zmq to a specific version. You could try removing that pin.
Did you remember to run after the change doom sync -u? (the -u is important here).
An update: after pulling from master, I'm getting a new error, namely #494. I have no idea what's going on here. @nnicandro could you shed some light into this? Is Emacs 30 not supported yet? I could simply roll back to the commit that works, no problem. I am just curious.
I recently updated a bunch of packages and have run into this error as well.
With my setup, the following minimal example
(package-initialize)
(require 'zmq)
(require 'jupyter-org-client)
(jupyter-run-repl "python3" nil nil 'jupyter-org-client)
results in error in process filter: zmq--subprocess-filter: Error in ZMQ subprocess: error, ("Invalid signature (08e97a35441b3635965c8f89598244282895018dcbc2887aa3ecbd374c0eeaa4) running on emacs -q.
Versions:
- emacs 29.1, fresh install
- org-9.6.9
- zmq-20230608.1856
- jupyter-20230925.2149
@cutonbuminband If this is an issue with jupyter, you should require the jupyter instead of jupyter-org-client?
@pati-ni you're right. I had the other one because that's what was in my backtraces when I started with an org file. The following gives the same error, and removes the org dependency:
(package-initialize)
(require 'zmq)
(require 'jupyter)
(jupyter-run-repl "python3")
Update: I'm still seeing this problem in version 20231016.146. When I add the line (setq jupyter-use-zmq nil) to my init, and to my minimal example, things work fine. That makes me suspect that the issue is somewhere in the $\texttt{zmq} \leftrightarrow \texttt{emacs-jupyter}$ interface.
Looking at the traceback it runs all the way from the (byte-compiled) elisp, through the zmq C module and to the jupyter python code. That's a lot of moving parts. I'm happy to provide more information, or to dig through things myself, but I don't know where to start.
could be related to https://discourse.jupyter.org/t/debugger-warning-it-seems-that-frozen-modules-are-being-used-python-3-11-0/16544/2
at least, after setting export PYDEVD_DISABLE_FILE_VALIDATION=1
the invalid signature error disappeared. I am still unable to start the kernel, though
Been following this over past few mos running Emacs29 on Linux, which required pinning emacs-jupyter, mentioned in https://github.com/emacs-jupyter/jupyter/issues/464#issuecomment-1703430789
Recently I went back to current emacs-jupyter master, but got a different error, similar to https://github.com/emacs-jupyter/jupyter/issues/500. Fixing this requires downgrading jupyter_server along with related Python packages.
Hoping this further helps isolate recent issues with emacs-jupyter. Anyone else with similar experience?
I did the downgrade to 4551667 and I can confirm that this works on Linux with emacs 29.1, python3.11 and jupyter server 2.7.2
I have spent a bit time on this and have a theory as to what is going on (on my machine at least) when this is happening. Disclaimer: I have very little experience with emacs lisp and the emacs-jupyter code base, so I see this as a pointer for someone that knows what they are doing to come up with a proper diagnosis / solution.
When creating a new repl, emacs-jupyter comes through jupyter-session-with-random-ports, where it creates a kernel process to have it figure out connection details for it to use. It records said details, except for the key which it regenerates, then interrupts and deletes the process. But for me, that process stays running. Later, in jupyter--start-kernel-process, it tries to launch a new ipykernel it wants to keep with the connection details from above (minus the key). Since the process from above is still running the ports are in use and the ipykernel can't start properly (which the code seems to fail to detect, probably since the check on a running process is positive while the ipkernel is still starting up). However, there is an ipykernel listening on the correct ports, the one from jupyter-session-with-random-ports above, which will answer but expects a different key for signing; hence the invalid signature .
My hack for this is to insert the line:
(while (process-live-p process))
right before this line.
Hope this helps!
I can confirm that this works on the master branch (linux 6.7.4-1 with jupyter from with conda). It does seem however that the initial startup time is much longer?
Just FYI: I just ran into this issue as well (Arch Linux, Emacs 29.4, all packages installed from (M)ELPA, updated to their latest versions, i.e., nothing pinned). The solution/work-around posted above by @deepestthought42 works for me.
( Windows 10, Python 3.12) hello, has anyone of you been through this zmq error: [I 2024-08-06 18:57:37.702 ServerApp] Connecting to kernel f1c5d513-19ad-4283-a971-a90849930c2e. [W 2024-08-06 18:57:37.827 ServerApp] Got events for closed stream <zmq.eventloop.zmqstream.ZMQStream object at 0x000001CDAD8B2B10> [W 2024-08-06 18:57:37.827 ServerApp] Got events for closed stream <zmq.eventloop.zmqstream.ZMQStream object at 0x000001CDAD8CE270>
The kernel won't connect and when i try to restart kernel, it says "Kernel Unknown" even when ipykernel is there. Haven' found any successful solution for this yet. Version details: IPython : 8.26.0 ipykernel : 6.29.5 ipywidgets : 8.1.3 jupyter_client : 8.6.2 jupyter_core : 5.7.2 jupyter_server : 2.14.2 jupyterlab : 4.2.4 nbclient : 0.10.0 nbconvert : 7.16.4 nbformat : 5.10.4 notebook : 7.2.1 qtconsole : 5.5.2 traitlets : 5.14.3
(Windows 10, Python 3.10) Guys i figured it out for me, installing python 3.10 and the anaconda 3 and then installing jupyter notebook 6.5.7 in conda env from Anaconda navigator finally helped. Here i'm attaching the current version details for everyone's reference. Thank you for all your efforts as i read in other discussions of zmq related errors which helped me solve this out. jupyter --version Selected Jupyter core packages... IPython : 8.25.0 ipykernel : 6.28.0 ipywidgets : not installed jupyter_client : 7.4.9 jupyter_core : 5.7.2 jupyter_server : 2.14.1 jupyterlab : not installed nbclient : 0.8.0 nbconvert : 7.10.0 nbformat : 5.9.2 notebook : 6.5.7 qtconsole : not installed traitlets : 5.14.3