TLJH Bootstrap depends on umask set to 022 for successful install

[crutchfield]

Bug description

On a fresh install of tljh the server will not start for the admin (first user).

How to reproduce

1. Install tljh on Ubuntu 22.04
2. Sign into JupyterHub as administrator
3. See server will not start

Expected behaviour

Expected server to start for user.

Actual behaviour

Attempt to launch server times out.

Your personal set up

the-littlest-jupyterhub

• OS:

• Version(s): Ubuntu 22.04 [CIS Ubuntu Linux 22.04 LTS Benchmark - Level 1 version 1.0.0.13]
• Python 3.10

Full environment

# paste output of `pip freeze` or `conda list` here

Configuration

# jupyterhub_config.py
"""
JupyterHub config for the littlest jupyterhub.
"""
import os
from glob import glob
from tljh import configurer
from tljh.config import CONFIG_DIR, INSTALL_PREFIX, USER_ENV_PREFIX
from tljh.user_creating_spawner import UserCreatingSpawner
from tljh.utils import get_plugin_manager
c = get_config()  # noqa
c.JupyterHub.spawner_class = UserCreatingSpawner
c.JupyterHub.cleanup_servers = False
c.JupyterHub.hub_port = 15001
c.TraefikProxy.should_start = False
dynamic_conf_file_path = os.path.join(INSTALL_PREFIX, "state", "rules", "rules.toml")
c.TraefikFileProviderProxy.dynamic_config_file = dynamic_conf_file_path
c.JupyterHub.proxy_class = "traefik_file"
c.SystemdSpawner.extra_paths = [os.path.join(USER_ENV_PREFIX, "bin")]
c.SystemdSpawner.default_shell = "/bin/bash"
c.SystemdSpawner.unit_name_template = "jupyter-{USERNAME}"
tljh_config = configurer.load_config()
configurer.apply_config(tljh_config, c)
pm = get_plugin_manager()
pm.hook.tljh_custom_jupyterhub_config(c=c)
extra_configs = sorted(glob(os.path.join(CONFIG_DIR, "jupyterhub_config.d", "*.py")))
for ec in extra_configs:
    load_subconfig(ec)

Logs

[welcome[bot]]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively. You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:

[SUNKENDREAMS]

Did you install on your own server or in 'the cloud'?

[crutchfield]

Did you install on your own server or in 'the cloud'?

I installed on an EC2 instance with a Ubuntu 22.04 image.

[MridulS]

Could you look at the logs https://tljh.jupyter.org/en/latest/troubleshooting/logs.html#jupyterhub-logs ? Does that give you any clues?

[crutchfield]

Could you look at the logs https://tljh.jupyter.org/en/latest/troubleshooting/logs.html#jupyterhub-logs ? Does that give you any clues?

It seems to me that permission on the /opt/tljh/user directory is not correct. I am attaching the install log, journal for services, and the transient service for the xadministrator. I am interested in the services extra lines "EnvironmentFile" and "Exec".

journal.jupyterhub.txt traefik.txt xadmin.service.txt xadmin.txt installer.txt

[crutchfield]

I did a fresh install again today using "https://tljh.jupyter.org/bootstrap.py" to install. Again the juypter-singleuser will not spawn a server for any users. I did a chown -R root:jupyterhub-user /opt/tljh and it works (i.e., spawns servers). This is almost certainly not the correct/final solution. But maybe helps identify the root cause.

[cmichal2]

I think I saw the same thing last week. I think the problem is the umask. In /etc/login.defs, there is:

#UMASK		022
UMASK		027

which I believe is the culprit. I didn't try reinstalling with a umask of 022, but I bet that would fix this problem. Instead, I did;

chmod o+r /opt/tljh/hub/* -R
chmod o+r /opt/tljh/user/* -R
find /opt/tljh/user -type d -exec chmod 755 {} +
find /opt/tljh/hub -type d -exec chmod 755 {} +

Perhaps the installer could explicitly set the umask before it puts files in place?

[crutchfield]

I think I saw the same thing last week. I think the problem is the umask. In /etc/login.defs, there is:

#UMASK		022
UMASK		027

which I believe is the culprit. I didn't try reinstalling with a umask of 022, but I bet that would fix this problem. Instead, I did;

chmod o+r /opt/tljh/hub/* -R
chmod o+r /opt/tljh/user/* -R
find /opt/tljh/user -type d -exec chmod 755 {} +
find /opt/tljh/hub -type d -exec chmod 755 {} +

Perhaps the installer could explicitly set the umask before it puts files in place?

I can verify our umask is 027 in the /etc/login.defs.

[crutchfield]

I am using CIS Ubuntu Linux 22.04 LTS Benchmark - Level 1 version 1.0.0.13 of the OS provided by the Center for Internet Security on an AWS EC2 instance. The umask is 027 by default instead of a common configuration of 022. It appears bootstrap.py depends on the umask to not be the more restrictive 027. I wrapped the bootstrap in a script setting the umask to 022 before calling bootstrap and the install is successful.

For example, `#!/bin/bash

umask 022 /usr/bin/python3 /tmp/bootstrap.py --admin xadministrator`

I suspect a better solution is to modify bootstrap.py to set the umask to 022 since it has a dependency on it to install properly.