feature request: support non-UTC timezone (was -- non-default time: Idle timeout triggering after 30s, making "-e COCALC_NO_IDLE_TIMEOUT=yes" necessary for operation)

[fsimula]

On a CentOS 8 box, instantiate a container with the latest CoCalc image: sudo docker run --name=cocalc_prova -d -v /opt/projects:/projects -v "/etc/localtime:/etc/localtime" -p 444:443 sagemathinc/cocalc:latest then create a .term and try editing it - the idle timeout triggers after at most 1 minute, stopping the project, even mid-typing.

The problem disappears instantiating the container and disabling the idle timeout with sudo docker run -e COCALC_NO_IDLE_TIMEOUT=yes --name=cocalc_prova -d -v /opt/projects:/projects -v "/etc/localtime:/etc/localtime" -p 444:443 sagemathinc/cocalc:latest but it seems there is something not working correctly, at least for images produced later than version tagged 6e341d886a40 on Docker Hub, which didn't have this problem.

[williamstein]

Why are you bind mounting /etc/localtime ? Do you see the problem if you don't do that?

[williamstein]

Another question: if instead of using a terminal, you edit a file in cocalc (i.e., via codemirror or jupyter), what happens?

[fsimula]

Why are you bind mounting /etc/localtime ? Do you see the problem if you don't do that?

The date reported by the date command in the terminal is one hour early without importing that file - now that you mentioned it, I tried without and the date is wrong but the timeout does not appear.

[fsimula]

Another question: if instead of using a terminal, you edit a file in cocalc (i.e., via codemirror or jupyter), what happens?

Same thing, of course when importing the /etc/localtime file.

[williamstein]

Unfortunately, we definitely haven't yet implemented support for the cocalc server having a clock that is not set to UTC, so I'm sure that's the source of this bug. I added a note in the title to clarify this.

[tornaria]

I was just bitten by this issue and it was so much pain!!! Check out https://github.com/sagemathinc/cocalc-docker/blob/master/README.md?plain=1#L69 where this is suggested. Maybe the suggestion can be reverted and replaced by a big fat warning instead?

Now, here's my story in case it helps someone:

1. I installed cocalc-docker back in august for a course I'm teaching this term
2. After it was installed and running, I realized the timezone was set to UTC inside the container, so I configured /etc/localtime inside the container.
3. This morning, the web server was down so I restarted the docker container, which had been running uninterrupted for 30+ days.
4. Student projects were restarting at precisely 60s. I configured student projects for always-run and the class went on fine.
5. I saw this ticket in the morning, but it didn't seem related -- after all, I never changed anything in the server since I first installed it in august!
6. I spent the whole afternoon debugging without luck. I came back to this ticket. I remembered I had configured localtime back in august.
7. I removed the symlink and restarted the docker container. Now everything seems back to normal.

The moral of the story is that the timezone is only used when starting the web server... That's what baffled me. The web server crash was caused by a project triggering OOM and I assumed that project was at fault for the timeout issue...

---

Regarding the suggestion in the README.md: it says there that one needs to setup both /etc/timezone and /etc/localtime. I only did the localtime part and not the timezone part. If the problem is only because of this mismatch then the suggestion is ok but a warning about the danger of the mismatch would be useful.

[tornaria]

The last_edited column in the projects table seems to be stored in local time without a timezone. The following code in src/packages/server/projects/control/stop-idle-projects.ts:

      query: `SELECT project_id, (EXTRACT(EPOCH FROM NOW() - last_edited))::FLOAT as idle_time, settings, run_quota
         FROM projects
         WHERE state ->> 'state' = 'running'`,

Is computing the difference between NOW() (which has timezone) to last_edited (in local time without timezone). So the computed difference will be offset by the local timezone (so timezones west of UTC will have a possibly negative timeout).

A (most probably bad) workaround would be to replace NOW() by LOCALTIMESTAMP. This will break on time zone changes (daylight savings, etc). Moreover, it's not even clear to me that updates the last_edited column always use localtime. The "proper" fix seems to be ensuring last_edited is always stored in UTC, and/or change the columnn to include a timezone.

Now I found a different workaround that seems to be working ok: instead of using /etc/localtime, I set the TZ environment variable in /etc/environment. Then projects seem to pick up the right timezone, but the hub seems to be running in UTC (at least the last_edited column looks ok).

[williamstein]

I'm certainly interested in supporting timezones properly. I just haven't been able to prioritize it, given it isn't needed for cocalc.com.

I've been running cocalc locally a lot on various computers I have (so directly on macOS, etc., with no vm) and I'm obviously hitting timezone issues as a result, since I'm not setting my laptop to UTC.

[tornaria]

Now I found a different workaround that seems to be working ok: instead of using /etc/localtime, I set the TZ environment variable in /etc/environment. Then projects seem to pick up the right timezone, but the hub seems to be running in UTC (at least the last_edited column looks ok).

As a matter of fact, setting TZ in /etc/environment seems to only affect ssh sessions, but not the cocalc linux terminal.

I guess users can just set TZ in .bashrc or in the project settings if they care about it.

[tornaria]

I've been running cocalc locally a lot on various computers I have (so directly on macOS, etc., with no vm) and I'm obviously hitting timezone issues as a result, since I'm not setting my laptop to UTC.

If running in docker, only the timezone inside the container matters.

In any case, kudos, since everything has been soooo smooth with the cocalc-docker (14 students, in 6 weeks of class this is the only issue we had so far).

[williamstein]

Thanks!! What functionality are you using? Have they tried out the new computational whiteboard yet? https://cocalc.com/features/whiteboard

[tornaria]

Thanks!! What functionality are you using? Have they tried out the new computational whiteboard yet? https://cocalc.com/features/whiteboard

Looks awesome! I had not noticed it, but I will tell the students, they may want to use it for their projects. We are using the course management with peer grading.