jupynium.nvim icon indicating copy to clipboard operation
jupynium.nvim copied to clipboard

Jupynium updates not reflected in browser without manual save when using SSH + WSL setup

Open DaitiDay opened this issue 2 months ago • 6 comments

First of all, I apologize if I used the wrong template, but since this is neither a bug nor a feature, I preferred to choose the free-form template to avoid any misunderstandings about the purpose of this issue.

I mainly work from my laptop (running Linux) and connect via SSH to my home PC. Specifically, the PC runs Windows, but I connect directly to WSL2.

My workflow is as follows:

  1. From the laptop, I open a file with nvim --listen localhost:<port> <file>
  2. Still from the laptop, I open an SSH session with port forwarding to the PC: ssh -X -R <port>:localhost:<port> <remote>
  3. On the remote PC, I start Jupynium: jupynium --nvim_listen_addr localhost:<port>

I had to add the flags --firefox_profiles_ini_path and --firefox_profile_name to the command to start Jupynium in order to avoid errors with environment variables. Furthermore, I had to use MOZ_HEADLESS=1 jupynium ... when I tried to launch Jupynium directly from the SSH session, since without I got the error:

selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 1

To be able to control the code from a browser, I did another SSH port forwarding, this time to port 8888 (ssh -L 8888:localhost:8888 <remote>).

The problem I encountered is that to see updated information in the browser, I have to save the file and then refresh the page.

Since I’ve never tried such a “complicated” setup before, I don’t know if this is normal or if I made any mistakes in the steps, or if there is something that could be improved/optimized.

Can someone provide feedback or advice on this?

I’m fully available if any clarifications/further information are needed.

DaitiDay avatar Nov 13 '25 17:11 DaitiDay

Okay, you should open jupynium locally most definitely. Because Jupynium is a browser, and you want to see the browser on your local machine.

But neovim & jupyter notebook can be anywhere. Just make jupynium connect to the right socket to neovim, and make it open the URL for the notebook wherever it is hosted.

Did I answer your confusion correctly, or is the issue different?

kiyoon avatar Nov 14 '25 01:11 kiyoon

That's exactly my issue, thank you for the reply. But the procedure is not clear to me. At the moment I have tried:

  1. Laptop -> nvim --listen localhost:18898
  2. Laptop -> ssh -L 18888:localhost:18888 <remote>
  3. via ssh -> conda activate <env>
  4. via ssh -> jupyter notebook --no-browser --port 18888
  5. Laptop -> jupynium --nvim_listen_addr 18898 --notebook_URL localhost:18888

The browser opens, but as soon as I JupyniumStartSync, I get:

jupynium.selenium_helpers: 57 - - Time out waiting for page to load
...
jupynium.cmds.jupynium: 589 - - Uncaught exception occurred while processing events. Detaching nvim

and then the browser closes, and the notebook server is killed.

What am I missing?

Edit: I should mention that the conda env I would like to use is running inside WSL2 on windows host. This shouldn't be a problem since I can establish a connection and the port forwarding is working just fine (I can see and use the jupyter notebook form the browser), but it may help with the troubleshooting.

DaitiDay avatar Nov 14 '25 10:11 DaitiDay

The browser opens, and you can see the Jupyter Notebook, but in like 10 seconds it errors out with the timed out message?

The error is from this

def wait_until_notebook_list_loaded(driver: WebDriver, timeout: int = 10):
    """Wait until the Jupyter Notebook home page (list of files) is loaded."""
    try:
        WebDriverWait(driver, timeout).until(
            EC.presence_of_element_located(
                (By.CSS_SELECTOR, "#notebook_list > div > div > a > span")
            )
        )
    except TimeoutException:
        logger.exception("Timed out waiting for page to load")
        driver.quit()

and if your page is showing the file list (#notebook_list) it should pass that check.

kiyoon avatar Nov 14 '25 15:11 kiyoon

Wait, are you using nbclassic? Jupynium doesn't support Notebook 7 and the correct URL is localhost:18888/nbclassic with pip install nbclassic

kiyoon avatar Nov 14 '25 15:11 kiyoon

The browser opens on the file tree, the server attaches to nvim. When I launch JupyniumStartSync in neovim, I can see in jupynium logs:

jupynium.events_control:  212 - INFO - Event from nvim: Request(type='request', name='start_sync', args=[1, '', True, ['# %%', ''], 'python', '/home/daiti/tmp/test_jup/.venv'], response=<pynvim.msgpack_rpc.async_session.Response object at 0x7f9e727167b0>)

Then it hangs for around 10 seconds, and then

jupynium.selenium_helpers:   57 - ERROR - Timed out waiting for page to load
Traceback (most recent call last):
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/jupynium/selenium_helpers.py", line 51, in wait_until_notebook_list_loaded
    WebDriverWait(driver, timeout).until(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        EC.presence_of_element_located(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            (By.CSS_SELECTOR, "#notebook_list > div > div > a > span")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        )
        ^
    )
    ^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/selenium/webdriver/support/wait.py", line 138, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
Stacktrace:
RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8
WebDriverError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:202:5
NoSuchElementError@chrome://remote/content/shared/webdriver/Errors.sys.mjs:555:5
dom.find/</<@chrome://remote/content/shared/DOM.sys.mjs:136:16

urllib3.connectionpool:  868 - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9e7101c910>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/0d94d305-d65c-4c3d-9a54-ba785676a98d/element
urllib3.connectionpool:  868 - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9e7101cf50>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/0d94d305-d65c-4c3d-9a54-ba785676a98d/element
urllib3.connectionpool:  868 - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9e72718770>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/0d94d305-d65c-4c3d-9a54-ba785676a98d/element
jupynium.cmds.jupynium:  589 - ERROR - Uncaught exception occurred while processing events. Detaching nvim.
Traceback (most recent call last):
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
        (self._dns_host, self.port),
    ...<2 lines>...
        socket_options=self.socket_options,
    )
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
        conn,
    ...<10 lines>...
        **response_kw,
    )
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/connectionpool.py", line 493, in _make_request
    conn.request(
    ~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<6 lines>...
        enforce_content_length=enforce_content_length,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/connection.py", line 494, in request
    self.endheaders()
    ~~~~~~~~~~~~~~~^^
  File "/home/daiti/.local/share/uv/python/cpython-3.14.0-linux-x86_64-gnu/lib/python3.14/http/client.py", line 1333, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/daiti/.local/share/uv/python/cpython-3.14.0-linux-x86_64-gnu/lib/python3.14/http/client.py", line 1093, in _send_output
    self.send(msg)
    ~~~~~~~~~^^^^^
  File "/home/daiti/.local/share/uv/python/cpython-3.14.0-linux-x86_64-gnu/lib/python3.14/http/client.py", line 1037, in send
    self.connect()
    ~~~~~~~~~~~~^^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/connection.py", line 325, in connect
    self.sock = self._new_conn()
                ~~~~~~~~~~~~~~^^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
        self, f"Failed to establish a new connection: {e}"
    ) from e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f9e72718510>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/jupynium/cmds/jupynium.py", line 584, in main
    status, rpcrequest_event = process_events(nvim_info, driver)
                               ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/jupynium/events_control.py", line 222, in process_events
    status, request_event = process_request_event(nvim_info, driver, event)
                            ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/jupynium/events_control.py", line 468, in process_request_event
    start_sync_with_filename(
    ~~~~~~~~~~~~~~~~~~~~~~~~^
        bufnr,
        ^^^^^^
    ...<6 lines>...
        driver=driver,
        ^^^^^^^^^^^^^^
    )
    ^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/jupynium/events_control.py", line 316, in start_sync_with_filename
    new_btn = driver.find_element(By.ID, "new-buttons")
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/selenium/webdriver/remote/webdriver.py", line 926, in find_element
    return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})["value"]
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/selenium/webdriver/remote/webdriver.py", line 455, in execute
    response = cast(RemoteConnection, self.command_executor).execute(driver_command, params)
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/selenium/webdriver/remote/remote_connection.py", line 407, in execute
    return self._request(command_info[0], url, body=data)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/selenium/webdriver/remote/remote_connection.py", line 431, in _request
    response = self._conn.request(method, url, body=body, headers=headers, timeout=self._client_config.timeout)
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/_request_methods.py", line 143, in request
    return self.request_encode_body(
           ~~~~~~~~~~~~~~~~~~~~~~~~^
        method, url, fields=fields, headers=headers, **urlopen_kw
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/_request_methods.py", line 278, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/poolmanager.py", line 459, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/connectionpool.py", line 871, in urlopen
    return self.urlopen(
           ~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<13 lines>...
        **response_kw,
        ^^^^^^^^^^^^^^
    )
    ^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/connectionpool.py", line 871, in urlopen
    return self.urlopen(
           ~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<13 lines>...
        **response_kw,
        ^^^^^^^^^^^^^^
    )
    ^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/connectionpool.py", line 871, in urlopen
    return self.urlopen(
           ~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<13 lines>...
        **response_kw,
        ^^^^^^^^^^^^^^
    )
    ^
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
        method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
    )
  File "/home/daiti/tmp/test_jup/.venv/lib/python3.14/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=42455): Max retries exceeded with url: /session/0d94d305-d65c-4c3d-9a54-ba785676a98d/element (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9e72718510>: Failed to establish a new connection: [Errno 111] Connection refused'))
urllib3.connectionpool:  868 - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9e7267b2f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/0d94d305-d65c-4c3d-9a54-ba785676a98d/window/handles
urllib3.connectionpool:  868 - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9e726aff00>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/0d94d305-d65c-4c3d-9a54-ba785676a98d/window/handles
urllib3.connectionpool:  868 - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9e7107c380>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/0d94d305-d65c-4c3d-9a54-ba785676a98d/window/handles
jupynium.cmds.jupynium:  619 - INFO - Browser disconnected. Quitting Jupynium.
urllib3.connectionpool:  868 - WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9e7107cc00>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/0d94d305-d65c-4c3d-9a54-ba785676a98d
urllib3.connectionpool:  868 - WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9e7107cf30>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/0d94d305-d65c-4c3d-9a54-ba785676a98d
urllib3.connectionpool:  868 - WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9e7107d260>: Failed to establish a new connection: [Errno 111] Connection refused')': /session/0d94d305-d65c-4c3d-9a54-ba785676a98d
jupynium.cmds.jupynium:  161 - SUCCESS - Piecefully closed as the browser is closed.

As for the nbclassic, I wasn't using it. I tried launching jupynium with jupynium --nvim_listen_addr localhost:18898 --notebook_URL localhost:18888/nbclassic but when the browser opens I get the jupyter logo at the top of a blank page with a 404: Not Found You are requesting a page that does not exist! error in the middle

Edit: In the venv I use to launch jupynium I have notebook both nbclassic (and jupyter-console) installed.

DaitiDay avatar Nov 14 '25 16:11 DaitiDay

Can you capture the screen of your notebook? I think you're using Notebook 7 and not nbclassic.

kiyoon avatar Nov 15 '25 01:11 kiyoon

Image

DaitiDay avatar Nov 15 '25 08:11 DaitiDay

Ok, now everything is working as expected. I have no idea what caused the issue, but my best guess is that while I was checking every command and log, repeatedly stopping and re-running the same commands, I left one of them running in a tmux session I forgot about, which caused some kind of conflict.

For future reference, here is the step-by-step procedure to locally edit a file on pcA while using a virtual environment envB on a remote pcB via SSH:

  1. On pcA, open the file in neovim: nvim --listen <nvim_port> <file>
  2. On pcA, open an SSH connection to pcB with port forwarding for <jupyter_port>: ssh -L <jupyter_port>:localhost:8888 <remote-pcB>
  3. On pcB via the SSH session: conda activate <envB> (or whatever command you use to activate the virtual environment)
  4. On pcB via the SSH session: jupyter notebook --no-browser
  5. On pcA: jupynium --nvim_listen_addr localhost:<nvim_port> --notebook_URL "localhost:<jupyter_port>/nbclassic"
  6. Work on the file 😄

Thank you so much for helping me figure out how to make everything work together.

I would suggest updating the documentation/README to make it easier to follow.

I will close the issue today since there’s nothing else to do regarding jupynium, but I’d like to know: is there a way to get suggestions/autocomplete from the remote virtual environment while editing the file locally? It's kind of annoying to see all the Import <package> could not be resolved and other errors in the editor 🤔

DaitiDay avatar Nov 15 '25 10:11 DaitiDay

I'm glad it worked out, and thanks for summarising it! I'll consider updating it maybe just by linking your comment.

Regarding the remote virtual environment, I don't think there's a way. You have to either synchronise the environment better (but I know that between different OS it's not always possible) or just open the nvim on the remote machine.

kiyoon avatar Nov 15 '25 13:11 kiyoon

Quick update: I manage to solve the LSP issue:

  1. I switched from pyright to pylsp, and installed python-lsp-server in the remote venv
  2. On pcB I start the LSP server with pylsp --tcp --port <lsp_port> --host 127.0.0.1
  3. On pcA I installed netcat and added pylsp to my local nvim config setting a custom command:
require'lspconfig'.pylsp.setup{
    cmd = {"nc", "localhost", "<lsp_port>" },
}
  1. On pcA I port forwarded port <lsp_port> with ssh -L <lsp_port>:localhost:<lsp_port> <remote-pcB>
  2. Opening a python file has all the information of the remote LSP, therefore suggesion/autocompletions/etc are all working as intended
  3. I need to figure out how to switch between local and remote LSP depending on the file, but that shouldn't be that hard to do

Edit: Turns out, you can have both pyright and pylsp installed in nvim, install mason and then just LspStop pyright or LspStop pylsp depending on the file/project.

DaitiDay avatar Nov 15 '25 14:11 DaitiDay