zero-to-jupyterhub-k8s icon indicating copy to clipboard operation
zero-to-jupyterhub-k8s copied to clipboard

Resource documentation feedback

Open consideRatio opened this issue 4 years ago • 1 comments
trafficstars

Please use this GitHub issue to provide feedback on our documented resource requests!

Please accompany feedback with information about...

  • Helm chart version
  • Estimated simultaneous users during heavy load
  • Estimated maximum users spawning a server over a 15 minute duration

consideRatio avatar May 25 '21 10:05 consideRatio

Hello,

I'm deploying Jupyterhub on k8s for a machine learning course. We expect 10 000 users in total and about 200 daily users.

We used to have a monolithic setup based on https://github.com/jupyterhub/jupyterhub-deploy-docker :

  • Config : 128 CPU / 512 Go RAM
  • Max simultaneous users : ~500 over 15min

The new k8s setup :

  • Config :
    • Nodepool for JH (1 node : 2 CPU / 7 GO RAM)
    • Nodepool for Users (10-50 nodes : 4 CPU / 15 Go RAM)
  • Max simultaneous users : ~150 over 15min then JH crashes (503 and 429 errors)

I'm using a modified version of yuvipanda/hubtraf to run heavy machine learning code (requesting CPU and RAM)

The aim is to have a scalable setup that can reach 500 simultaneous users. For now I'm unable to reach that goal but I'll update this post when I find the bottleneck (it's probably due to few CPUs on the JH node).

brospars avatar Feb 02 '22 16:02 brospars