jupyter
jupyter copied to clipboard
emacs-jupyter
'Kernel did not respond to kernel-info request' when using ssh session.
I'm trying to connect to my pc at work through out login node (as a jumphost). This the block config I'm using:
#+BEGIN_SRC jupyter-julia :kernel julia-1.2 :session /ssh:workpc:julia
# some code
#+END_SRC
The session is spawned correctly (after setting (setq jupyter--debug t)) the output is:
executing Jupyter-Julia code block...
jupyter-start-kernel: default-directory = /ssh:workpc:
jupyter-start-kernel: Starting process with args "/bin/python3 -c from jupyter_client.kernelapp import main; main() --kernel=julia-1.2"
Tramp: Opening connection for workpc using ssh...
Tramp: Sending command ‘exec ssh -q -o ControlMaster=auto -o ControlPath='tramp.%C' -o ControlPersist=no -e none workpc’
Tramp: Waiting for prompts from remote shell...done
Tramp: Found remote shell prompt on ‘workpc’
Tramp: Opening connection for workpc using ssh...done
Launching julia-1.2 kernel process...done
Tramp: Inserting ‘/ssh:workpc:/run/user/10039/jupyter/kernel-0336e5aa-8bf6-4fe5-9eb0-932cdb035f11.json’...
Tramp: Encoding remote file ‘/ssh:workpc:/run/user/10039/jupyter/kernel-0336e5aa-8bf6-4fe5-9eb0-932cdb035f11.json’ with ‘base64 <%s’...done
Tramp: Decoding local file ‘/var/folders/0z/xyn859fx3vj4762qh24f8dq80000gn/T/tramp.3XGFeL.json’ with ‘base64-decode-region’...done
Tramp: Inserting ‘/ssh:workpc:/run/user/10039/jupyter/kernel-0336e5aa-8bf6-4fe5-9eb0-932cdb035f11.json’...done
SENDING: :kernel-info-request 886301a0-fe6d-4e5d-865b-b8027afa8bde nil
SENT: (:shell 886301a0-fe6d-4e5d-865b-b8027afa8bde)
Requesting kernel info...done
jupyter-kernel-info: Kernel did not respond to kernel-info request
The path to the session file is the correct one. My ssh config is:
Host A
HostName ...
User ...
Host workpc
HostName ...
ProxyJump A
User ...
ProxyCommand ssh -W %h:%p A
Perhaps the discussion in #72 can help you (see my comment at the end). It may have something to do with the way TRAMP calls Jupyter, as discussed in the referenced link.
I suffer from similar issues without JumpHost, even after following the advice of @arthurcgusmao.
My remote machine is just a simple conda installation.
Just to add to this issue: as @tpanum I tried following the advice of @arthurcgusmao and, also, still ran into issues.
Eventually I managed to figure out that for, some reason that I do not understand, when making the ssh process run in the background, i.e. using -f, the process would terminate almost immediately. It's worth noting that I'm ssh-forwarding through another server, i.e. using ProxyCommand in ~/.ssh/config with ssh -q other-host nc target-host 22. So maybe this is causing issues?
I fixed this by changing jupyter-make-ssh-tunnel to
(defun jupyter-make-ssh-tunnel (lport rport server remoteip)
(or remoteip (setq remoteip "127.0.0.1"))
(start-process
"jupyter-ssh-tunnel-%s-%s" lport rport nil ;; <= added the ports in case this name needs to be unique (I don't know)
"ssh"
;; Run in background
;; "-f" <= COMMENTED OUT THIS LINE
;; Wait until the tunnel is open
"-o ExitOnForwardFailure=yes"
;; Local forward
"-L" (format "127.0.0.1:%d:%s:%d" lport remoteip rport)
server
;; Close the tunnel if no other connections are made within 60
;; seconds
"sleep 60"))
Now, I'm not certain if this is the a hack or the actual fix. The reason why I tried it was because I connected the subprocces to a buffer so I could see why the processed terminated immediately. With -f the buffer told me that the subprocesses terminated, but nothing more. So I removed it to get the output upon termination; lo and behold, now the processes didnt' terminate immediately and everything worked fine :confused:
In addition, I would be awesome if this would produce an error. One approach could be to connect the process to a buffer by replacing the nil argument with the buffer-name above, and then check the buffer to see if the processes exited for some reason. At least it helps identifying where the issue is.
And thank you so much for this package! At this point I can't live without it.
It is no surprise, but I just wanted to mention the same issue with essentially equivalent debug output arises with a remote host followed by a container using docker-tramp.
In particular,
:PROPERTIES:
:header-args: :results output verbatim replace :session /ssh:cloudmachine|docker:containeroncloudmachine:sessionname
:END:
#+BEGIN_SRC jupyter-python :kernel python3
x = 'foo'
y = 'bar'
x + ' ' + y
#+END_SRC
yields
executing Jupyter-Python code block...
jupyter-start-kernel: default-directory = /ssh:cloudmachine|docker:containeroncloudmachine:
jupyter-start-kernel: Starting process with args "/bin/python3 -c from jupyter_client.kernelapp import main; main() --kernel=python3"
Tramp: Opening connection for containeroncloudmachine using docker...
Tramp: Sending command ‘exec ssh -q -e none cloudmachine’
Tramp: Waiting for prompts from remote shell...done
Tramp: Found remote shell prompt on ‘cloudmachine’
Tramp: Sending command ‘exec docker exec -it containeroncloudmachine sh’
Tramp: Waiting for prompts from remote shell...done
Tramp: Found remote shell prompt on ‘containeroncloudmachine’
Tramp: Opening connection for containeroncloudmachine using docker...done
Launching python3 kernel process...done
Tramp: Inserting ‘/ssh:cloudmachine|docker:containeroncloudmachine:/home/jovyan/.local/share/jupyter/runtime/kernel-fc5b0ea7-f553-4725-aa59-32829d356665.json’...
Tramp: Encoding remote file ‘/ssh:cloudmachine|docker:containeroncloudmachine:/home/jovyan/.local/share/jupyter/runtime/kernel-fc5b0ea7-f553-4725-aa59-32829d356665.json’ with ‘base64 <%s’...done
Tramp: Decoding local file ‘/var/folders/1d/wtzfcz5s4x98nbkdx9g5ss3c0000gn/T/tramp.krOJmR.json’ with ‘base64-decode-region’...done
Tramp: Inserting ‘/ssh:cloudmachine|docker:containeroncloudmachine:/home/jovyan/.local/share/jupyter/runtime/kernel-fc5b0ea7-f553-4725-aa59-32829d356665.json’...done
SENDING: :kernel-info-request ae928b51-f755-441e-a250-8a08c58d734d nil
SENT: (:shell ae928b51-f755-441e-a250-8a08c58d734d)
Requesting kernel info...done
jupyter-kernel-info: Kernel did not respond to kernel-info request
jupyter-kernel-info is the function from which the error Kernel did not respond to kernel-info request arises (see line 2066 of jupyter-client.el).
The stack trace for jupyter-kernel-info is
Debugger entered--entering a function:
jupyter-kernel-info(#<jupyter-org-client jupyter-org-client-1fe73d7aa114>)
jupyter--error-if-no-kernel-info(#<jupyter-org-client jupyter-org-client-1fe73d7aa114>)
jupyter-start-new-kernel("julia-1.5" jupyter-org-client)
jupyter-run-repl("julia-1.5" nil nil jupyter-org-client)
#f(compiled-function (session kernel) "Initiate a client connected to a remote kernel process." #<bytecode 0x1fe7435018f5>)(#s(org-babel-jupyter-remote-session :name "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." :connect-repl-p nil) "julia-1.5")
apply(#f(compiled-function (session kernel) "Initiate a client connected to a remote kernel process." #<bytecode 0x1fe7435018f5>) (#s(org-babel-jupyter-remote-session :name "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." :connect-repl-p nil) "julia-1.5"))
#f(compiled-function (&rest args) #<bytecode 0x1fe743520a15>)(#s(org-babel-jupyter-remote-session :name "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." :connect-repl-p nil) "julia-1.5")
apply(#f(compiled-function (&rest args) #<bytecode 0x1fe743520a15>) (#s(org-babel-jupyter-remote-session :name "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." :connect-repl-p nil) "julia-1.5"))
#f(compiled-function (&rest cnm-args) #<bytecode 0x1fe7430d03fd>)()
#f(compiled-function (cl--cnm session kernel) "Rename the returned client's REPL buffer to include SESSION's name.\nAlso set `jupyter-include-other-output' to nil for the session so\nthat output produced by other clients do not get handled by the\nclient." #<bytecode 0x1fe7434f577d>)(#f(compiled-function (&rest cnm-args) #<bytecode 0x1fe7430d03fd>) #s(org-babel-jupyter-remote-session :name "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." :connect-repl-p nil) "julia-1.5")
apply(#f(compiled-function (cl--cnm session kernel) "Rename the returned client's REPL buffer to include SESSION's name.\nAlso set `jupyter-include-other-output' to nil for the session so\nthat output produced by other clients do not get handled by the\nclient." #<bytecode 0x1fe7434f577d>) #f(compiled-function (&rest cnm-args) #<bytecode 0x1fe7430d03fd>) (#s(org-babel-jupyter-remote-session :name "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." :connect-repl-p nil) "julia-1.5"))
#f(compiled-function (&rest args) #<bytecode 0x1fe743520a41>)(#s(org-babel-jupyter-remote-session :name "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." :connect-repl-p nil) "julia-1.5")
apply(#f(compiled-function (&rest args) #<bytecode 0x1fe743520a41>) #s(org-babel-jupyter-remote-session :name "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." :connect-repl-p nil) "julia-1.5")
org-babel-jupyter-initiate-client(#s(org-babel-jupyter-remote-session :name "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." :connect-repl-p nil) "julia-1.5")
org-babel-jupyter-initiate-session-by-key("/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "both") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:eval . "never-export") (:async . "no") (:session . "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt...") (:kernel . "julia-1.5")))
#f(compiled-function (&optional session params) "Initialize a Jupyter SESSION according to PARAMS." #<bytecode 0x1fe7439bd0c1>)("/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "both") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:eval . "never-export") (:async . "no") (:session . "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt...") (:kernel . "julia-1.5")))
apply(#f(compiled-function (&optional session params) "Initialize a Jupyter SESSION according to PARAMS." #<bytecode 0x1fe7439bd0c1>) ("/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "both") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:eval . "never-export") (:async . "no") (:session . "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt...") (:kernel . "julia-1.5"))))
org-babel-jupyter-initiate-session("/ssh:notebooks-vm.us-central1-f.quarere|docker:klt..." ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "both") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:eval . "never-export") (:async . "no") (:session . "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt...") (:kernel . "julia-1.5")))
org-babel-execute:jupyter-julia("x = \"foo\"\ny = \"bar\"\nprintln(x)\nprintln(y)" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "both") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:eval . "never-export") (:async . "no") (:session . "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt...") (:kernel . "julia-1.5")))
#f(compiled-function (&optional arg info params) "Execute the current source code block.\nInsert the results of execution into the buffer. Source code\nexecution and the collection and formatting of results can be\ncontrolled through a variety of header arguments.\n\nWith prefix argument ARG, force re-execution even if an existing\nresult cached in the buffer would otherwise have been returned.\n\nOptionally supply a value for INFO in the form returned by\n`org-babel-get-src-block-info'.\n\nOptionally supply a value for PARAMS which will be merged with\nthe header arguments specified at the front of the source code\nblock." (interactive nil) #<bytecode 0x1fe742a390dd>)(nil nil nil)
ob-async-org-babel-execute-src-block(#f(compiled-function (&optional arg info params) "Execute the current source code block.\nInsert the results of execution into the buffer. Source code\nexecution and the collection and formatting of results can be\ncontrolled through a variety of header arguments.\n\nWith prefix argument ARG, force re-execution even if an existing\nresult cached in the buffer would otherwise have been returned.\n\nOptionally supply a value for INFO in the form returned by\n`org-babel-get-src-block-info'.\n\nOptionally supply a value for PARAMS which will be merged with\nthe header arguments specified at the front of the source code\nblock." (interactive nil) #<bytecode 0x1fe742a390dd>) nil)
apply(ob-async-org-babel-execute-src-block #f(compiled-function (&optional arg info params) "Execute the current source code block.\nInsert the results of execution into the buffer. Source code\nexecution and the collection and formatting of results can be\ncontrolled through a variety of header arguments.\n\nWith prefix argument ARG, force re-execution even if an existing\nresult cached in the buffer would otherwise have been returned.\n\nOptionally supply a value for INFO in the form returned by\n`org-babel-get-src-block-info'.\n\nOptionally supply a value for PARAMS which will be merged with\nthe header arguments specified at the front of the source code\nblock." (interactive nil) #<bytecode 0x1fe742a390dd>) nil)
org-babel-execute-src-block(nil)
(cond ((eq type 'headline) (cond ((memq (and (boundp 'org-goto-map) org-goto-map) (current-active-maps)) (org-goto-ret)) ((and (fboundp 'toc-org-insert-toc) (member "TOC" (org-get-tags))) (toc-org-insert-toc) (message "Updating table of contents")) ((string= "ARCHIVE" (car-safe (org-get-tags))) (org-force-cycle-archived)) ((or (org-element-property :todo-type context) (org-element-property :scheduled context)) (org-todo (if (eq (org-element-property :todo-type context) 'done) (or (car ...) 'todo) 'done)))) (org-update-checkbox-count) (org-update-parent-todo-statistics) (if (and (fboundp 'toc-org-insert-toc) (member "TOC" (org-get-tags))) (progn (toc-org-insert-toc) (message "Updating table of contents"))) (let* ((beg (if (org-before-first-heading-p) (line-beginning-position) (save-excursion (org-back-to-heading) (point)))) (end (if (org-before-first-heading-p) (line-end-position) (save-excursion (org-end-of-subtree) (point)))) (overlays (condition-case nil (progn (overlays-in beg end)) (error nil))) (latex-overlays (cl-find-if #'(lambda ... ...) overlays)) (image-overlays (cl-find-if #'(lambda ... ...) overlays))) (+org--toggle-inline-images-in-subtree beg end) (if (or image-overlays latex-overlays) (org-clear-latex-preview beg end) (org--latex-preview-region beg end)))) ((eq type 'clock) (org-clock-update-time-maybe)) ((eq type 'footnote-reference) (org-footnote-goto-definition (org-element-property :label context))) ((eq type 'footnote-definition) (org-footnote-goto-previous-reference (org-element-property :label context))) ((memq type '(timestamp planning)) (org-follow-timestamp-link)) ((memq type '(table-row table)) (if (org-at-TBLFM-p) (org-table-calc-current-TBLFM) (condition-case nil (progn (save-excursion (goto-char (org-element-property :contents-begin context)) (org-call-with-arg 'org-table-recalculate (or arg t)))) (error nil)))) ((eq type 'table-cell) (org-table-blank-field) (org-table-recalculate arg) (if (and (string-empty-p (string-trim (org-table-get-field))) (and (boundp 'evil-local-mode) evil-local-mode)) (progn (evil-change-state 'insert)))) ((eq type 'babel-call) (org-babel-lob-execute-maybe)) ((eq type 'statistics-cookie) (save-excursion (org-update-statistics-cookies arg))) ((memq type '(inline-src-block src-block)) (org-babel-execute-src-block arg)) ((memq type '(latex-environment latex-fragment)) (org-latex-preview arg)) ((eq type 'link) (let* ((lineage (org-element-lineage context '(link) t)) (path (org-element-property :path lineage))) (if (or (equal (org-element-property :type lineage) "img") (and path (image-type-from-file-name path))) (+org--toggle-inline-images-in-subtree (org-element-property :begin lineage) (org-element-property :end lineage)) (org-open-at-point arg)))) ((org-element-property :checkbox (org-element-lineage context '(item) t)) (let ((match (and (org-at-item-checkbox-p) (match-string 1)))) (org-toggle-checkbox (if (equal match "[ ]") '(16))))) (t (if (or (org-in-regexp org-ts-regexp-both nil t) (org-in-regexp org-tsr-regexp-both nil t) (org-in-regexp org-link-any-re nil t)) (call-interactively #'org-open-at-point) (+org--toggle-inline-images-in-subtree (org-element-property :begin context) (org-element-property :end context)))))
(let* ((context (org-element-context)) (type (org-element-type context))) (while (and context (memq type '(verbatim code bold italic underline strike-through subscript superscript))) (setq context (org-element-property :parent context) type (org-element-type context))) (cond ((eq type 'headline) (cond ((memq (and (boundp ...) org-goto-map) (current-active-maps)) (org-goto-ret)) ((and (fboundp 'toc-org-insert-toc) (member "TOC" (org-get-tags))) (toc-org-insert-toc) (message "Updating table of contents")) ((string= "ARCHIVE" (car-safe (org-get-tags))) (org-force-cycle-archived)) ((or (org-element-property :todo-type context) (org-element-property :scheduled context)) (org-todo (if (eq ... ...) (or ... ...) 'done)))) (org-update-checkbox-count) (org-update-parent-todo-statistics) (if (and (fboundp 'toc-org-insert-toc) (member "TOC" (org-get-tags))) (progn (toc-org-insert-toc) (message "Updating table of contents"))) (let* ((beg (if (org-before-first-heading-p) (line-beginning-position) (save-excursion ... ...))) (end (if (org-before-first-heading-p) (line-end-position) (save-excursion ... ...))) (overlays (condition-case nil (progn ...) (error nil))) (latex-overlays (cl-find-if #'... overlays)) (image-overlays (cl-find-if #'... overlays))) (+org--toggle-inline-images-in-subtree beg end) (if (or image-overlays latex-overlays) (org-clear-latex-preview beg end) (org--latex-preview-region beg end)))) ((eq type 'clock) (org-clock-update-time-maybe)) ((eq type 'footnote-reference) (org-footnote-goto-definition (org-element-property :label context))) ((eq type 'footnote-definition) (org-footnote-goto-previous-reference (org-element-property :label context))) ((memq type '(timestamp planning)) (org-follow-timestamp-link)) ((memq type '(table-row table)) (if (org-at-TBLFM-p) (org-table-calc-current-TBLFM) (condition-case nil (progn (save-excursion (goto-char ...) (org-call-with-arg ... ...))) (error nil)))) ((eq type 'table-cell) (org-table-blank-field) (org-table-recalculate arg) (if (and (string-empty-p (string-trim (org-table-get-field))) (and (boundp 'evil-local-mode) evil-local-mode)) (progn (evil-change-state 'insert)))) ((eq type 'babel-call) (org-babel-lob-execute-maybe)) ((eq type 'statistics-cookie) (save-excursion (org-update-statistics-cookies arg))) ((memq type '(inline-src-block src-block)) (org-babel-execute-src-block arg)) ((memq type '(latex-environment latex-fragment)) (org-latex-preview arg)) ((eq type 'link) (let* ((lineage (org-element-lineage context '... t)) (path (org-element-property :path lineage))) (if (or (equal (org-element-property :type lineage) "img") (and path (image-type-from-file-name path))) (+org--toggle-inline-images-in-subtree (org-element-property :begin lineage) (org-element-property :end lineage)) (org-open-at-point arg)))) ((org-element-property :checkbox (org-element-lineage context '(item) t)) (let ((match (and (org-at-item-checkbox-p) (match-string 1)))) (org-toggle-checkbox (if (equal match "[ ]") '(16))))) (t (if (or (org-in-regexp org-ts-regexp-both nil t) (org-in-regexp org-tsr-regexp-both nil t) (org-in-regexp org-link-any-re nil t)) (call-interactively #'org-open-at-point) (+org--toggle-inline-images-in-subtree (org-element-property :begin context) (org-element-property :end context))))))
Printing the value of the client variable from inside edebug on jupyter-kernel-info yields
;; client ;;pp-eval-last-sexp
#s(jupyter-org-client
(#<finalizer>)
jupyter--clients "idle" 1 #s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data
())
nil #s(jupyter-channel-ioloop-comm
(#s(hash-table size 1 test eql weakness value rehash-size 1.5 rehash-threshold 0.8125 data
(t #0)))
#s(jupyter-zmq-channel-ioloop
(#<finalizer>)
#<process zmq> nil
((send
((channel jupyter-channel)
msg-type msg msg-id)
((list 'sent
(oref channel type)
(jupyter-send channel msg-type msg msg-id))))
(stop-channel
(type)
((let
((channel
(object-assoc type :type jupyter-channel-ioloop-channels)))
(when
(and channel
(jupyter-channel-alive-p channel))
(jupyter-stop-channel channel))
(list 'stop-channel type))))
(start-channel
((channel jupyter-channel)
endpoint)
((when
(jupyter-channel-alive-p channel)
(jupyter-stop-channel channel))
(oset channel endpoint endpoint)
(let
((identity
(jupyter-session-id jupyter-channel-ioloop-session)))
(jupyter-start-channel channel :identity identity))
(list 'start-channel
(oref channel type)))))
((setq jupyter-channel-ioloop-session
(jupyter-session :id "38bcac68-f74f-4bd2-b1e7-998df7c14c4f" :key "a1369b21-aa7ff7834dd1f3fa5f7108e7"))
(require 'jupyter-channel-ioloop)
(require 'jupyter-zmq-channel-ioloop)
(push 'jupyter-zmq-channel-ioloop--recv-messages jupyter-ioloop-post-hook)
(cl-loop for channel in
'(:shell :stdin :iopub)
unless
(object-assoc channel :type jupyter-channel-ioloop-channels)
do
(push
(jupyter-zmq-channel :session jupyter-channel-ioloop-session :type channel)
jupyter-channel-ioloop-channels)))
((mapc #'jupyter-stop-channel jupyter-channel-ioloop-channels)))
#s(jupyter-hb-channel :hb #s(jupyter-session
(:shell_port 60543 :iopub_port 52071 :stdin_port 46591 :control_port 37599 :hb_port 49243 :ip "127.0.0.1" :key "a1369b21-aa7ff7834dd1f3fa5f7108e7" :transport "tcp" :signature_scheme "hmac-sha256" :kernel_name "julia-1.5")
"38bcac68-f74f-4bd2-b1e7-998df7c14c4f" "a1369b21-aa7ff7834dd1f3fa5f7108e7")
"tcp://127.0.0.1:49243" #<user-ptr ptr=0x6000002f88a0 finalizer=0x10e782ed0> 10 ignore t t)
jupyter-zmq-channel-ioloop #s(jupyter-session
(:shell_port 60543 :iopub_port 52071 :stdin_port 46591 :control_port 37599 :hb_port 49243 :ip "127.0.0.1" :key "a1369b21-aa7ff7834dd1f3fa5f7108e7" :transport "tcp" :signature_scheme "hmac-sha256" :kernel_name "julia-1.5")
"38bcac68-f74f-4bd2-b1e7-998df7c14c4f" "a1369b21-aa7ff7834dd1f3fa5f7108e7")
(:stdin #s(jupyter-proxy-channel "tcp://127.0.0.1:46591" t)
:shell #s(jupyter-proxy-channel "tcp://127.0.0.1:60543" t)
:iopub #s(jupyter-proxy-channel "tcp://127.0.0.1:52071" t)))
#s(jupyter-session
(:shell_port 60543 :iopub_port 52071 :stdin_port 46591 :control_port 37599 :hb_port 49243 :ip "127.0.0.1" :key "a1369b21-aa7ff7834dd1f3fa5f7108e7" :transport "tcp" :signature_scheme "hmac-sha256" :kernel_name "julia-1.5")
"38bcac68-f74f-4bd2-b1e7-998df7c14c4f" "a1369b21-aa7ff7834dd1f3fa5f7108e7")
#s(hash-table size 65 test equal rehash-size 1.5 rehash-threshold 0.8125 data
())
#s(jupyter-kernel-process-manager
(#<finalizer>)
jupyter--kernel-managers #s(jupyter-command-kernel
(#<finalizer>)
("julia-1.5" "/ssh:notebooks-vm.us-central1-f.quarere|docker:klt-notebooks-vm-cjme:/home/jovyan/.local/share/jupyter/kernels/julia-1.5" :argv
["/usr/bin/julia" "-i" "--startup-file=yes" "--color=yes" "--project=@." "/home/jovyan/.julia/packages/IJulia/tOM8L/src/kernel.jl" "{connection_file}"]
:env nil :display_name "Julia 1.5.1" :language "julia" :interrupt_mode "signal" :metadata nil)
#s(jupyter-session
(:shell_port 60543 :iopub_port 52071 :stdin_port 46591 :control_port 37599 :hb_port 49243 :ip "127.0.0.1" :key "a1369b21-aa7ff7834dd1f3fa5f7108e7" :transport "tcp" :signature_scheme "hmac-sha256" :kernel_name "julia-1.5")
"38bcac68-f74f-4bd2-b1e7-998df7c14c4f" "a1369b21-aa7ff7834dd1f3fa5f7108e7")
#<process jupyter-kernel-julia-1.5>)
#s(jupyter-zmq-channel :control #s(jupyter-session
(:shell_port 60543 :iopub_port 52071 :stdin_port 46591 :control_port 37599 :hb_port 49243 :ip "127.0.0.1" :key "a1369b21-aa7ff7834dd1f3fa5f7108e7" :transport "tcp" :signature_scheme "hmac-sha256" :kernel_name "julia-1.5")
"38bcac68-f74f-4bd2-b1e7-998df7c14c4f" "a1369b21-aa7ff7834dd1f3fa5f7108e7")
"tcp://127.0.0.1:37599" #<user-ptr ptr=0x6000002f8900 finalizer=0x10e782ed0>))
#<buffer *jupyter-kernel-client*> nil "null" nil nil nil)
When I check all the existing kernel files in the remote container, I find one whose kernel ID is different ( 14238987-f1b4-4049-982a-94012ddb7087 )from what is contained in the client variable ( 38bcac68-f74f-4bd2-b1e7-998df7c14c4f ), but whose key and various ports are all correct.
/docker:klt-notebooks-vm-cjme:/home/jovyan/ #$ cat .local/share/jupyter/runtime/kernel-14238987-f1b4-4049-982a-94012ddb7087.json
{
"shell_port": 60543,
"iopub_port": 52071,
"stdin_port": 46591,
"control_port": 37599,
"hb_port": 49243,
"ip": "127.0.0.1",
"key": "a1369b21-aa7ff7834dd1f3fa5f7108e7",
"transport": "tcp",
"signature_scheme": "hmac-sha256",
"kernel_name": "julia-1.5"
}
This suggests that the problem might be related to the fact that the kernel ID is not being captured or updated accurately. There is no kernel with an ID equivalent to the one that appears in the client variable. It is not clear to me where the value of the kernel ID that appears in the client variable printed above is being updated.
It is also possible to see this issue with the kernel ID from jupyter-make-client (see L141 in jupyter-kernel-manager.el) when calling make-instance class for class jupyter-org-client which derives from jupyter-repl-client which derives from jupyter-kernel-client. The session is defined as an element of jupyter-kernel-client at L215 of jupyter-client.el.
Running
(with-slots (kernel) manager
(oref kernel session))
on a defined instance of manager we can see the jupyter-session information including the kernel ID
#s(jupyter-session :conn-info (:shell_port 35979 :iopub_port 59059 :stdin_port 44689 :control_port 58895 :hb_port 43877 :ip "127.0.0.1" :key #1="032bc810-7888808c1a03df393f8e98b4" :transport "tcp" :signature_scheme "hmac-sha256" :kernel_name "python3") :id "a0ec867b-2706-40db-bcfc-97773655652e" :key #1#)
where the kernel ID is again incorrect and the correct value (obtained by looking at the kernel files in .local/share/jupyter/runtime happens to be 6e4f9818-311c-47fa-9ce2-6f35726d9592.
@cameronraysmith I have tried to list the kernels on a server that is accessed via TRAMP as follows:
(setq jupyter-current-server (car (jupyter-servers)))
(jupyter-server-list-kernels jupyter-current-server)
This fails with a message:
url-retrieve-internal: Bad url: /ssh:notebooks-vm.us-central1-f.quarere%7Cdocker:klt-notebooks-vm-cjme:notebooks01/login
@nnicandro Could you please help us understand how to check whether the remote kernel is known to Emacs? From the above, it's not clear that emacs-jupyter is fully aware of the available kernels (though we can confirm that the kernels are produced on the server).
We are using these parameters inside Org mode to define the connection:
* Run a jupyter kernel in a remote container
:PROPERTIES:
:header-args: :results output verbatim replace :session /ssh:notebooks-vm.us-central1-f.quarere|docker:klt-notebooks-vm-cjme:notebooks01 :exports both :eval never-export
:END:
As others have indicated above, jupyter-kernel-info seems not to be able to find the kernel either and fails with the message indicated by @sebastianpech in the title.
Hi all, have you guys figured out anything more about this issue? I'd be happy to run some tests if you guys have ideas.
Thanks @dangom. @holtzermann17 and I have not had a chance to test this since our original postings, but perhaps some of the commits by @nnicandro from Jan-Apr 2021 have had an impact. I will try to test this again within the next few weeks and identify whether or not I am still experiencing the same collection of issues described above.
after sys reinstall on remote I have a similar problem...
jupyter-kernel-info: Kernel did not respond to kernel-info request finalizer failed: (cl-no-primary-method jupyter-kill-kernel #s(jupyter-kernel-process-manager (#<finalizer used>) jupyter--kernel-managers #s(jupyter-spec-kernel (#<finalizer> #<finalizer used>)
The connection supposed to be established with the help of remote_ikernel, as I was changing and checking settings, kernel config files, deleting old ssh-keys, confusingly enough sometimes the connection to the remote jupyter-python kernel was established without any problem, so I thought that I have eliminated the cause of the problem, but what at first seemed to me to be in a random fashion, after emacs restart or system reboot the kernel-connection could not be established again and because this happen a couple of times it was driving me mad xD
Now, I haven't identified the root cause, but I have established that it works without any problems if I first do (find-file "/ssh:username@remote:/home/username/"). After the ssh connection is established then jupyter is able to connect (in org-mode my source block header contains only the remote ikernel directory name, as :session and that directory contains JSON config generated by remote_ikernel script).
it looks like tramp or jupyter-kernel have problem establishing the ssh connection (that worked without any hassle before remote system was reinstalled). It wouldn't be a solution, but a workaround can be to advise one of the emacs-jupyter functions to find-file on remote and close it immediately after ssh connection is successfully established.
but I would appreciate any suggestions on how to diagnose this problem...
