kubespawner icon indicating copy to clipboard operation
kubespawner copied to clipboard

JupyterHub Singleuser Port Routing Issue After Restart

Open IMSUVEN opened this issue 1 year ago • 3 comments

Bug description

When customizing the singleuser port (e.g., setting it to 9999), the routing works correctly, directing traffic to port 9999. However, after a JupyterHub restart, the routing becomes erroneous and redirects to port 8888 instead.

How to reproduce

  1. use a custom singleuser port (e.g., 9999) in jupyterhub.
  2. Start Jupyter notebook and verify that the routing works as expected.
  3. Restart JupyterHub.
  4. Observe that the routing now incorrectly directs to port 8888 instead of the custom port (9999).

Expected behaviour

After a JupyterHub restart, the routing should continue to direct traffic to the custom singleuser port specified during configuration.

Actual behaviour

After a JupyterHub restart, the routing that was previously working correctly (directing traffic to the custom singleuser port, e.g., 9999) becomes erroneous. Instead of continuing to route to the specified custom port, it now incorrectly redirects to port 8888.

Your personal set up

  • Deployment Method: zero-to-jupyterhub-k8s
  • Chart name: jupyterhub
  • Chart version: 3.2.1
  • Chart config: singleuser.profileList[0].port=9999

Others

  1. Observation:

    • After starting a JupyterHub singleuser pod, the database (servers table) correctly records a port value of 9999.
    • However, upon restarting JupyterHub, this port value is changed to 8888 in the database.
    • Simultaneously, the configurable-http-proxy receives a request to modify the routing to port 8888.
    • Eventually, JupyterHub becomes unable to connect to the singleuser pod via port 8888, leading to the cleaning of the singleuser pod.
  2. Hypothesis:

    • The bug seems to occur during JupyterHub's initial startup, where it incorrectly resets all singleuser routes to port 8888 without referencing the port settings stored in the database.

IMSUVEN avatar Mar 27 '24 03:03 IMSUVEN

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively. welcome You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:

welcome[bot] avatar Mar 27 '24 03:03 welcome[bot]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively. welcome You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:

welcome[bot] avatar Mar 27 '24 09:03 welcome[bot]

It seems related to this

https://github.com/jupyterhub/kubespawner/blob/becea538dd075b063964123034111aa6463f79ac/kubespawner/spawner.py#L1924-L1959

IMSUVEN avatar Apr 07 '24 11:04 IMSUVEN

@IMSUVEN are you find solution?

BioQwer avatar Aug 25 '25 15:08 BioQwer

@IMSUVEN are you find solution?

I simply handled this issue using random ports and then overwrote spawner.py through k8s.

IMSUVEN avatar Aug 26 '25 14:08 IMSUVEN

did you save port in db ?

BioQwer avatar Aug 26 '25 14:08 BioQwer

i used random ports too, but on hub restart i lose all running labs.

BioQwer avatar Aug 26 '25 14:08 BioQwer

i used random ports too, but on hub restart i lose all running labs.

I overwrote the file /usr/local/lib/python3.11/site-packages/kubespawner/spawner.py and in the init function of KubeSpawner, I made the following changes:

            if self.port == 0:
                # Our default port is 8888
                self.port = random.randint(20000, 25000)

I configured the database for JupyterHub. Based on the above changes, each initial user will be assigned a random port number, and JupyterHub will store the port number in the database.

IMSUVEN avatar Aug 28 '25 02:08 IMSUVEN