encode-universal-time does not correctly adjust for time zone

[CK-DE]

I tested with:

Clozure Common Lisp Version 1.12 (v1.12) WindowsX8664

I am located in time zone MEST, which is one hour east of GMT, i.e. -1 in Common Lisp terms, and we currently have a daylight saving time:

CL-USER> local-time:*default-timezone*
#<TIMEZONE LMT CEST CET CEST CET CEMT CEMT CEST CET>
CL-USER> (get-decoded-time)
25
4
13
8
6
2021
1
T
-1

Consider the time 01:00:01 on Jan 1, 1976 -- in my current time zone:

CL-USER> (encode-universal-time 1 0 1 1 1 1976)
2398287601
CL-USER> (decode-universal-time (encode-universal-time 1 0 1 1 1 1976))
1
0
1
1
1
1976
3
T
-1

This is wrong, since on Jan 1, 1976 we didn't have a daylight saving time. Already the encoded time is wrong, it must be 2398291201.

Now consider what happens if I explicitly specify my current time zone:

CL-USER> (encode-universal-time 1 0 1 1 1 1976 -1)
2398291201
CL-USER> (decode-universal-time (encode-universal-time 1 0 1 1 1 1976 -1))
1
0
2
1
1
1976
3
T
-1

The encoded time is correct, but the decoded time should be 1 0 1 1 1 1976 3 NIL -1, since on Dec 1 we did have daylight saving time

LispWorks and SBCL do this correctly (tested on Windows).

[metayan]

Confirming that this is Windows-specific. On Linux and Darwin it behaves correctly.

@CK-DE : In the last example, to specify your current time zone, you probably should use (decode-universal-time (encode-universal-time 1 0 1 1 1 1976 -1) -1) The '-1' at the end of 'encode-universal-time' is the time zone of the "timestamp". Also, you probably mean "on Dec 1 we did not have daylight saving time".

https://github.com/Clozure/ccl/blob/master/lisp-kernel/windows-calls.c#L718-L730

[informatimago]

Le 08/06/2021 à 13:45, CK-DE a écrit :

I tested with:

|Clozure Common Lisp Version 1.12 (v1.12) WindowsX8664|

I am located in time zone MEST, which is one hour east of GMT, i.e. -1 in Common Lisp terms, and we currently have a daylight saving time:

Well, no. https://en.wikipedia.org/wiki/Central_European_Summer_Time

CEST IS the daylight saving timezone of CET. MEST is an alias for CEST.

CET is UTC+1 (-1 in CL) CEST is UTC+2 (-2 in CL)

If you are in MEST = CEST, then you are in UTC+2, CL -2, not CL -1 !

encode-universal-time:

encode-universal-time converts a time from Decoded Time format to a universal time.

If time-zone is supplied, no adjustment for daylight savings time is performed.

decode-universal-time:

Returns the decoded time represented by the given universal time.

If time-zone is not supplied, it defaults to the current time zone adjusted for daylight saving time. If time-zone is supplied, daylight saving time information is ignored. The daylight saving time flag is nil if time-zone is supplied.

Now, ignoring the ambiguity of not specifying what should happen in encode-universal-time if time-zone IS NOT supplied, and assuming that it means that it defaults to the current time zone adjusted for daylight saving time,like for decode-universal-time, there remains the BIG QUESTION:

 What daylight saving time?

The potential daylight saving time of the CURRENT time, or the potential daylight saving time at the time specified for encoding?

AFAIK, on January 1st 1976, there was no daylight saving time in effect in the Central European Time Zone, even if eg. in France, summer time was introduced in 1974.

I think there may be some confusion here, because in some countries, the timezone was shifted in winter, while in others it was shifted in summer.

So for example, in France we have "heure d'été", and the daylight saving is obtained in winter, were the timezone had been shifted so that more daylight was available in the end of afternoon, for enterprises to save on electricity and heating. (Of course, people weren't too happy to wake up in the night and arrive at the school or at work while still in the night! What kind of saving is this? (well, it was a saving for the corporations)).

But the English definition of Daylight Saving Time, is that of a Summer Time, ie shifting of timezone in Summer. See: https://en.wikipedia.org/wiki/Daylight_saving_time

Things of course are made more confusing since now we have merged time-zone for big areas in Europe, covering several countries, some of them had shifted time in winter, while others had shifted time in summer, originally. But they were now all in synch.

So, remember, daylight saving time = summer time.

|CL-USER> local-time:default-timezone #<TIMEZONE LMT CEST CET CEST CET CEMT CEMT CEST CET> CL-USER> (get-decoded-time) 25 4 13 8 6 2021 1 T -1 |

Consider the time 01:00:01 on Jan 1, 1976 -- in my current time zone:

|CL-USER> (encode-universal-time 1 0 1 1 1 1976) 2398287601 CL-USER> (decode-universal-time (encode-universal-time 1 0 1 1 1 1976)) 1 0 1 1 1 1976 3 T -1 |

This is wrong, since on Jan 1, 1976 we didn't have a daylight saving time. Already the encoded time is wrong, it must be 2398291201.

cl-user> local-time:default-timezone #<timezone PMT WEST WET WEST WET CET CEST CEST WEMT CET CEST CET> cl-user> (encode-universal-time 1 0 1 1 1 1976) 2398291201 cl-user> (encode-universal-time 1 0 1 1 1 1976 0) 2398294801 cl-user> (encode-universal-time 1 0 1 1 1 1976 -1) 2398291201 cl-user> (encode-universal-time 1 0 1 1 1 1976 -2) 2398287601

So, without a timezone, encode-universal-time uses CET instead CEST, which IMO seems perfectly correct, given that the date encoded was in CET, not in CEST. Accessorily, CET is DST = NIL, which is the case, since January 1st is in winter, not in summer.

cl-user> (encode-universal-time 1 0 1 1 7 1976) 2414012401 cl-user> (encode-universal-time 1 0 1 1 7 1976 0) 2414019601 cl-user> (encode-universal-time 1 0 1 1 7 1976 -1) 2414016001 cl-user> (encode-universal-time 1 0 1 1 7 1976 -2) 2414012401 cl-user>

As you can see, if you encode a summer date, encode-universal-time without a timezone uses the summer time CEST (ie DST = T).

Now consider what happens if I explicitly specify my current time zone:

|CL-USER> (encode-universal-time 1 0 1 1 1 1976 -1) 2398291201 CL-USER> (decode-universal-time (encode-universal-time 1 0 1 1 1 1976 -1)) 1 0 2 1 1 1976 3 T -1 |

The encoded time is correct, but the decoded time should be 1 0 1 1 1 1976 3 NIL -1, since on Dec 1 we did have daylight saving time

As you can see, this is consistent with decode-universal-time:

cl-user> (decode-universal-time (encode-universal-time 1 0 1 1 1 1976)) 1 0 1 1 1 1976 3 nil -1 cl-user> (decode-universal-time (encode-universal-time 1 0 1 1 7 1976)) 1 0 1 1 7 1976 3 t -1 cl-user>

the winter date is decoded with dst = NIL in CET, while the summer date is decoded with dst = T in CEST.

LispWorks and SBCL do this correctly (tested on Windows).

Well if they don't give the same results, I'd say they have a bug. We can't prove it formally (because the standard is not a formal document, see the above mentionned ambiguities and interpretations). But in the real world ccl encode-univeral-time and decode-universal-time give the results corresponding to the reality.

-- Pascal Bourguignon

[CK-DE]

Oh, yes of course I meant "on Dec 1 we did not have daylight saving time", sorry.

And yes, theoretically I could explicitly specify my timezone in the call to encode-universal-time, but this would require me to fumble out the timezone I am in (which is easy using get-decoded-time) and to figure out whether we have daylight saving time at the time specified (which is hard for me in the general case if the implementation doesn't do this correctly). I just used 2398291201 because this is used as an example in CLtL2.

So, my point was just to report that CCL's implementation of encode-universal-time is buggy, at least on Windows.

The LOCAL-TIME library on the other hand does seem to do the encoding and decoding correctly on CCL.

[informatimago]

Yes, sorry, I didn't try it on MS-Windows, but on macOS. At least, it is correct on macOS.

[CK-DE]

Pascal,

You are right with your comment about MEST. I find it quite difficult not to be confusing when talking about timezones. What I meant is that I live one timezone east of GMT, which is called Zone 1 on time zone maps. Currently, we have "Sommerzeit" in Germany, so our time is UTC+2, which is -2 in Common Lisp terms, as your wrote above. But I think it makes sense to distingiush between time differences because of location differences and those because of politically decided time shifting like "heure d'été". (Well, timezones are politically decided as well. For example, personally I don't think there is any resonable solution in the current European debate about daylight saving time without splitting Zone 1 (or moving its boundaries).)

The answer to your BIG QUESTION of what daylight saving time to use when encoding a specific time and date is easy for me: always use the one in effect at the specified dated to be encoded, in the current time zone. I have to admit that in CLHS it is not 100 % clear what is meant by "time zone adjusted for daylight saving time", but from my point of view this is the only sensible interpretation. Otherwise I as a programmer have to know about the switching dates for daylight saving time, something the implementation needs to be aware of anyway. Consider for example the follwowing times: 12:00:00 Oct 30, 2021 and 12:00:00 Oct 30, 2022. Oct 30, 2021 is a Sunday and DST will have been switched off in the previous night. Oct 30, 2022 though is a Saturday and DST is about to be switched off in the following night. In LispWorks on Windows I correctly get:

CMI 83 > (decode-universal-time (encode-universal-time 0 0 12 30 10 2021))
0
0
12
30
10
2021
5
T
-1
CMI 84 > (decode-universal-time (encode-universal-time 0 0 12 30 10 2022))
0
0
12
30
10
2022
6
NIL
-1

But in CCL on Windows I get:

CL-USER> (decode-universal-time (encode-universal-time 0 0 12 30 10 2021))
0
0
12
30
10
2021
5
T
-1
CL-USER> (decode-universal-time (encode-universal-time 0 0 12 30 10 2022))
0
0
12
30
10
2022
6
T
-1

Which doesn't make any sense.

Incidentally, summer time was introduced in France in 1976 (and in Germany in 1980). Its quite interesting what we can do with the LOCAL-TIME library:

LOCAL-TIME> *default-timezone*
#<TIMEZONE LMT CEST CET CEST CET CEMT CEMT CEST CET>
LOCAL-TIME> 
(let ((*default-timezone*
       (find-timezone-by-location-name "Europe/Paris")))
  (setf *timestamp* (encode-timestamp 0 0 0 12 1 7 1976))
  (format-timestring t *timestamp*))
1976-07-01T12:00:00.000000+02:00
"1976-07-01T12:00:00.000000+02:00"
LOCAL-TIME> *timestamp*
@1976-07-01T11:00:00.000000+01:00

This is really helpful, and impossible to do in Common Lisp without reinventing something like LOCAL-TIME, even if encode-universal-time was implemented correctly.

[CK-DE]

Or even concider the following, which is really great:

LOCAL-TIME> *default-timezone*
#<TIMEZONE LMT CEST CET CEST CET CEMT CEMT CEST CET>
LOCAL-TIME> 
(let ((*default-timezone*
       (find-timezone-by-location-name "Europe/Busingen")))
  (setf *timestamp* (encode-timestamp 0 0 0 12 1 7 1980))
  (format-timestring t *timestamp*))
1980-07-01T12:00:00.000000+01:00
"1980-07-01T12:00:00.000000+01:00"
LOCAL-TIME> *timestamp*
@1980-07-01T13:00:00.000000+02:00

Büsingen is a little German enclave in Switzerland near Schaffhausen which didn't switch to DST in 1980 as the rest of Germany did. I am sure you can find even more interesting examples for France.

[gmpalter]

Actually, the problem is that, apparently, DECODE-UNIVERSAL-TIME doesn't properly determine if DST was in effect for the time passed to it. It was always using the current time to make that decision instead.