lighthouse
lighthouse copied to clipboard
sigp
Logging timestamps changed from local time to UTC
Description
In Lighthouse v2.1.4 and later, logging timestamps changed from being in local time to UTC. This was due to an update to our logging library slog-term in a routine dependency bump PR (#3039). Upstream slog-term set all timestamps to UTC to prevent a race condition in one of their dependencies.
We are following https://github.com/slog-rs/term/pull/44 for a resolution, and would prefer not to revert to the previous racey version of slog-term.
@michaelsproul
Maybe you could just disable slog-term logging the time altogether, at least for systemd logs? If i remember right in systemd logs before v2.4.1 there was only the time by systemd. Now time gets logged twice on the same log line, once by systemd and once by lighthouse.
We have an issue open to track that feature here: https://github.com/sigp/lighthouse/issues/3101
I hope that something can be done about this. I much prefer having log files in my local time zone.
We have opted to use UTC by default as most events a relative in time and this helps debugging logs across multiple timezones.
fyi this decision is pretty non-standard. most other applications will use the timezone of the machine (which could be in UTC for situations like what you describe).
I was advocating for keeping UTC because I've found it's nice to not have to worry about what timezones users are using when they send me logs. This is mostly useful for comparing log timestamps against the starts of specific slots (e.g. from a tool like https://slots.symphonious.net/).
I realise this is a bit selfish on my part though, and I could deal with localtime logs. It's usually possible to infer the timezone, and doing a bit of manual conversion between user timezone <> UTC is bearable.
UTC has other benefits (no DST), but maybe it should be up to the user to opt-in, rather than us forcing it on them. As you say, many operators will know about running servers on UTC, and we could encourage community guides like Somer's & CoinCashew to recommend UTC as well.
Yes, this decision was assuming that there were no users that wanted local time. If users are interested in using local time we can re-open this issue and add the option for localtime or option for UTC.