lima
lima copied to clipboard
lima-vm
Clockdrift in the created guest
Description
Running Lima via Colima (https://github.com/abiosoft/colima/), I have occasional clock drift issues. I am unable to find a cause for this or even a way to reproduce it. It seems to be after I lock my laptop and get back the next morning, but only once in a while (every 10 days or so) and not consistently every day.
Versions:
❯ uname -a
Darwin Dirks-MacBook-Pro.local 21.4.0 Darwin Kernel Version 21.4.0: Mon Feb 21 20:36:53 PST 2022; root:xnu-8020.101.4~2/RELEASE_ARM64_T8101 arm64
❯ lima --version
limactl version 0.9.2
❯ colima version
colima version 0.3.4
git commit: 5a4a70481ca8d1e794677f22524e3c1b79a9b4ae
runtime: docker
arch: aarch64
client: v20.10.14
server: v20.10.11
docker run -it --rm --cap-add SYS_TIME --privileged -e ALLOW_CIDR=0.0.0.0/0 -p 123:123/udp geoffh1977/chrony
Writing New Config File
2022-05-07T08:00:43Z chronyd version 4.1 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP -SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 -DEBUG)
2022-05-07T08:00:43Z Could not read valid frequency and skew from driftfile /var/lib/chrony/chrony.drift
2022-05-07T08:00:43Z Initial frequency -100000.000 ppm
2022-05-07T08:00:47Z System's initial offset : 978.140307 seconds slow of true (step)
2022-05-07T08:00:52Z Backward time jump detected!
2022-05-07T08:02:00Z Selected source 162.159.200.1 (pool.ntp.org)
2022-05-07T08:02:00Z System clock wrong by 978.685231 seconds
2022-05-07T08:18:18Z System clock was stepped by 978.685231 seconds
2022-05-07T08:02:57Z Backward time jump detected!
2022-05-07T08:02:57Z Can't synchronise: no selectable sources
2022-05-07T08:05:07Z Selected source 94.198.159.15 (pool.ntp.org)
2022-05-07T08:05:07Z System clock wrong by 977.327132 seconds
2022-05-07T08:21:24Z System clock was stepped by 977.327132 seconds
2022-05-07T08:05:06Z Backward time jump detected!
2022-05-07T08:05:06Z Can't synchronise: no selectable sources
References https://github.com/abiosoft/colima/issues/274, https://github.com/lima-vm/lima/pull/490
Possibly alpine-lima is missing an NTP server ? I thought that lima was adding a RTC in qemu ?
For Ubuntu (systemd) I think it is being handled by systemd-timesyncd, checking with /dev/rtc0
Possibly alpine-lima is missing an NTP server ? I thought that lima was adding a RTC in qemu ?
It does indeed (in #490). For some reason I remembered that it would sync time with the host instead. So it looks like the RTC gets out of sync as well.
I'm seeing something similar in a different configuration (details at #939).
The issue still seems to exist. Any timeframe regarding a solution because recently I ran into that issue upon validating a JWT token (not before / expires etc.). the time difference was actually only about 0.4 seconds but that can be enough for its timestamps to not match anymore and validation to fail. It also seems to exist upon using macOS HyperKit (new virtualization framework).
using busybox as ntp (as mentioned in the article referenced in the comment https://github.com/lima-vm/lima/issues/850#issuecomment-1121980785) seems to work best. there still is a bit of drift but it seems to be the lowest one for now.
Created 2 VMs yesterday, woke up to way more drammatic shift in time
Wed Oct 19 04:16:54 UTC 2118
and
Wed Dec 15 06:03:40 UTC 2213
The worst thing, for some reason I cannot even set the date manually
hwclock --hctosys
hwclock: settimeofday() failed: Invalid argument
date -s '2023-09-01 01:00:00'
date: cannot set date: Invalid argument
Fri Sep 1 01:00:00 UTC 2023
To fix this issue. I have rebooted both VMs, that sync the dates, and I did after
timedatectl set-ntp no
apt install ntp
Hope that is going to fix this issue for now.
I am on lima 0.17.2, and using ubuntu 23.04/release-20230810/ubuntu-23.04-server-cloudimg-arm64.img
Running with vz on macOS Sonoma.
Working around (on Colima):
colima ssh -- sudo date -s @$(TZ=UTC date "+%s")
It's only accurate to the second, that's enough for my use case.
