Questions about kernels created by enterprise_gateway reading data

[kakaluoteasd]

Hello! I used enterpise_gateway in kubernetes, and now I have a problem. When I only used jupyterLab, I mounted data to jupyterLab with Persistent Volume. And when a kernel was started to execute code, as the kernel and the server was on the same node and container, the kernel could read the data with the path such as "/data/input" directly. However when I used enterprise_gateway to create kernels in kubernetes, the jupyter server I start and the kernel created by enterprise_gateway might in different nodes. And I mounted my data to jupyter server with Persistent Volume in order to see these data from frontend pages. In this way, when I wanted to execute code with the function of reading data, I used the path just as before such as"/data/input". And when the kernel started to execute the code, it couldn't find the data with the path, because there was no data on the kernels node.
Is there any suggestions to solve this problem? Thanks a lot!

[kevin-bates]

Hi @kakaluoteasd.

Yeah, you'll need to mount the same volume on the kernel pod. If the volume is the same across all kernel launches, you should be able to adjust the kernel-pod.yaml.j2 file corresponding to the desired kernel. (Note that this file is located in the scripts sub-directory for each kernelspec.

You might find it helpful to mount the kernelspecs (via NFS or however) in your EG instance so that you can edit these entries from outside the EG pod while troubleshooting.

Also keep in mind that any variances in the mount point names, like user names or something that varies per user, can be expressed via KERNEL_-prefixed environment variables from the jupyterlab server side. Any envs in that process will automatically flow to EG such that they are available for use in the kernel-pod template.

This portion of the docs may provide some additional insight: https://jupyter-enterprise-gateway.readthedocs.io/en/latest/kernel-kubernetes.html#unconditional-volume-mounts

There have been other issues in which this has been discussed and implemented, so search issues may also prove helpful.

I hope that helps.

[kakaluoteasd]

Hi @kevin-bates. Thanks for your detailed explanations and suggestions. Since I'm a beginner to the Jupyter ecosystem, I often confused by some every easy questions.
I have tried what you said to adjust the kernel-pod.yaml.j2 to mount the volume on the kernel pod and it worked great! However，when I was trying to add KERNEL_-prefixed environment variables in the kernel.json file, an error occured when I started a kernel. The error was showed as this: Error attempting to connect to Gateway server url 'http://124.70.1.193:80'. Ensure gateway url is valid and the Gateway instance is running.

Is there anything wrong with to add KERNEL_-prefixed environment variables in the kernel.json file? I add the environment variables just as below.

Thanks a lot for your help again!

[kevin-bates]

Hi @kakaluoteasd. The values for each of your envs need to be strings. This issue may help with the formatting: https://github.com/jupyter/enterprise_gateway/issues/753

Note that if you're actually going to update the kernel.json, you could probably just update the kernel-pod.yaml.j2 script directly. Then, for any "variables" (perhaps like /user1) let that be conveyed via a KERNEL_FOO env that comes from the Hub/Notebook server and let the kernel-pod.yaml reference that variable via {kernel_foo}.