Time zone information appears to be ignored

[waynew]

Version information.

I've tried with both the current head (4.7.0-4-ge1b5) as well as 4.7.0 found in brew. Both exhibit the same behavior.

Bug description.

When importing an event on a calendar with a bunch of timezone data/shifts, calcurse ignores this information.

Reproduce.

cat <<EOF >minimal.ics
BEGIN:VCALENDAR
METHOD:PUBLISH
VERSION:2.0
X-WR-CALNAME:Calendar
BEGIN:VTIMEZONE
TZID:Pacific Standard Time
BEGIN:STANDARD
DTSTART:16010101T020000
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=11
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=2SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
DESCRIPTION:Example
SUMMARY:Example
DTSTART;TZID=Pacific Standard Time:20201110T081500
DTEND;TZID=Pacific Standard Time:20201110T140000
DTSTAMP:20201111T161048Z
RRULE:FREQ=WEEKLY;UNTIL=20201112T161500Z;INTERVAL=1;BYDAY=TU,WE;WKST=SU
END:VEVENT
END:VCALENDAR
env TZ=America/Chicago calcurse -i minimal.ics
env TZ=America/Chicago calcurse -Q

The existing output is:

11/11/20:
 - 02:15 -> 08:00
	Example

But that's the wrong information. I'm honestly not sure - I thought it was just translating to UTC but:

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//calcurse//NONSGML v4.7.0-4-ge1b5//EN
BEGIN:VEVENT
DTSTART:20201110T021500
DURATION:P0DT5H45M0S
RRULE:FREQ=WEEKLY;UNTIL=20201112T021500;BYDAY=TU,WE
SUMMARY:Example
DESCRIPTION:Example
END:VEVENT
END:VCALENDAR

That looks like it's just subtracting 6h (which is the current UTC offset for America/Chicago). Ah... I see:

env TZ=America/Chicago date --date="2020-11-11 02:15 UTC"
Tue 10 Nov 2020 08:15:00 PM CST

So it looks like it's reading the time of the appointment (8:15, Pacific) and translating that to UTC. But it's not displaying in the current timezone, or rather perhaps it's treating the DTSTART as the agnostic timezone.

Expected Behavior.

This event should appear as starting 10:15AM, CST (America/Chicago). Not sure if this is technically the appointment getting the wrong DTSTART when it's imported, or if it's being stored as expected but just displayed incorrectly.

[lhca]

When importing an event on a calendar with a bunch of timezone data/shifts, calcurse ignores this information.

Have you read these: https://github.com/lfos/calcurse/issues/323#issuecomment-718876253 and https://github.com/lfos/calcurse/issues/323#issuecomment-719908117?

This event should appear as starting 10:15AM, CST (America/Chicago).

Dates and times are always stored and displayed as local times.

[waynew]

I read through that thread a little, but not super close. I might be misunderstanding the comments, but that second comment linked made it sound like calcurse is just straight up ignoring TZ info.

Some further experimentation also suggests that TZ info is getting discarded:

env TZ=America/Chicago calcurse -i minimal.ics
env TZ=etc/UTC calcurse -i minimal.ics
env TZ=America/Chicago calcurse -Q
env TZ=etc/UTC calcurse -Q

Both -Q commands display:

11/11/20:
 - 02:15 -> 08:00
	Example
 - 08:15 -> 14:00
	Example

That doesn't seem right to me at all. I had been trying to use -c to read ical files and thought I was just borked because my file was in the wrong format but I exported from calcurse and that file failed, too. I realized that there must be a different file format, and a bit of digging:

cat ~/.calcurse/apts                                  (28s 623ms)
11/10/2020 @ 02:15 -> 11/10/2020 @ 08:00 {1W -> 11/12/2020 w2 w3} >551e0edc71d01b8d5ea138174039f87eda0d6019 |Example
11/10/2020 @ 08:15 -> 11/10/2020 @ 14:00 {1W -> 11/12/2020 w2 w3} >551e0edc71d01b8d5ea138174039f87eda0d6019 |Example

Looks like in fact the timezone information is discarded. A more reliable approach would be to store the time in UTC and convert from UTC (which does still have leap seconds, but at least there's no DST) to local time on display.

[lhca]

Expected Behavior.

This event should appear as starting 10:15AM, CST (America/Chicago).

So it will, if you correct start and end time to

DTSTART;TZID=America/Los_Angeles:20201110T081500
DTEND;TZID=America/Los_Angeles:20201110T140000

I read through that thread a little, but not super close.

Maybe you should - and write a little less.

[TrevCan]

Hey, I also have the same issue. I recently imported a calendar with appointments (all of them in Mexico City Time (same as CDT). Below is a sample. after running calcurse -P to purge all data I ran calcurse -i cal-fr.ics but was still getting the wrong time. It's supposed to be 7:00 - 8:00 but it says 2:00 - 3:00

the file has specified the Mexico City Timezone. Attached to this issue is the file. cal-fr.ics.txt (I am aware that it ends in .txt, I needed to rename it to .txt or a supported file extension by GitHub.)

Got any ideas on how to fix this ?

[lhca]

Hey, I also have the same issue.

Yours is not the topic discussed above, but one of the problems discussed in #323, in particular see https://github.com/lfos/calcurse/issues/323#issuecomment-719908117 and https://github.com/lfos/calcurse/issues/323#issuecomment-719923566.

Your import file contains

BEGIN:VCALENDAR
METHOD:PUBLISH
PRODID:Microsoft Exchange Server 2010
VERSION:2.0
X-WR-CALNAME:Calendar
BEGIN:VEVENT
RRULE:FREQ=WEEKLY;UNTIL=20211231T130000Z;INTERVAL=1;BYDAY=MO,TU,WE,TH,FR;WK
 ST=SU
UID:040000008200E00074C5B7101A82E00800000000F63FBBF3C48CD701000000000000000
 0100000004342AAFF07957F43A9A34078A4B00DA2
SUMMARY:Françaisv
DTSTART;TZID=Central Standard Time (Mexico):20210809T070000
DTEND;TZID=Central Standard Time (Mexico):20210809T080000
CLASS:PUBLIC
PRIORITY:5
DTSTAMP:20210809T024151Z
TRANSP:OPAQUE
STATUS:CONFIRMED
SEQUENCE:0
LOCATION:https://zoom.us/j/ignorethisurl
X-MICROSOFT-CDO-APPT-SEQUENCE:0
X-MICROSOFT-CDO-BUSYSTATUS:BUSY
X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY
X-MICROSOFT-CDO-ALLDAYEVENT:FALSE
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-INSTTYPE:1
X-MICROSOFT-DONOTFORWARDMEETING:FALSE
X-MICROSOFT-DISALLOW-COUNTER:FALSE
END:VEVENT
END:VCALENDAR

but the time zone information "TZID=Central Standard Time (Mexico)" is Microsoft's and not recognized by calcurse. Unix-like systems have the TZ database and calcurse converts all times to local times. Try manually to edit the file and change the time zone information (twice) to "TZID=America/Mexico_City" and import it.

[TrevCan]

Thanks, worked great!

[machinedgod]

Hey all - sorry for resurrecting the old thread about the topic that's not even related to the header, but I wrote a tiny hack/script that automates what @lhca suggested as a solution , so I thought I'd share it here for anyone else who stumbles along.

Basically, test whether the file is coming from Microsoft, then apply a series of sed queries to replace the Microsoft standard with the tz database. I've chained it as a mailcap for mutt, and it works well.

You will need windowsZones.xml from here: https://github.com/unicode-org/cldr/blob/main/common/supplemental/windowsZones.xml

Remeber to adjust windowszones variable in the beginning.

#!/bin/sh

file=$1
windowszones="$HOME/bin/windowsZones.xml"
grep --silent 'PRODID:Microsoft' "$file"

if [ $? == 0 ]; then
	originaltz=$(sed --regexp-extended --silent 's|^TZID:(.+)$|\1|p' "$file")
	mappedtz=$(grep "$originaltz" "$windowszones" \
				| head -n1 \
				| sed --regexp-extended --silent 's|^.+type="([^"]+)".+$|\1|p')

	sed --regexp-extended \
		--expression="s|^TZID:.+$|TZID:$mappedtz|" \
		--expression="s|TZID=[^:]+|TZID=$mappedtz|" \
		"$file"
else
	cat "$file"
fi

[Moult]

@machinedgod that's a great solution! I can confirm it works on my .ics file coming from some Microsoft platform somewhere.

Can I ask why doesn't calcurse just implement this feature when it detects a timezone that it can't find in the Unix timezone database? Can't it fallback to this mapped TZ list? Surely I'm not the only person who is facing problems with Microsoft ics imports and missing meetings and so on?

[EpicVoyage]

@machinegod's script works well to solve this issue for ICS files containing a single timezone. My import has more than one timezone due to the locations of several people who are adding events, so I am going to submit this modification for that scenario. Please note that my version will modify your ICS file in-place.

#!/bin/sh

# TODO: Download https://github.com/unicode-org/cldr/blob/main/common/supplemental/windowsZones.xml
# then update the windowzones variable with its local location.
file=$1
windowszones="$HOME/data/windowsZones.xml"
grep --silent 'PRODID:Microsoft' "$file"

if [ $? == 0 ]; then
	sed --regexp-extended --silent 's|^TZID:(.+)\s+$|\1|p' "$file" | while read originaltz; do
		mappedtz=$(grep "$originaltz" "$windowszones" \
				| head -n1 \
				| sed --regexp-extended --silent 's|^.+type="([^"]+)".+$|\1|p')

		sed -i --regexp-extended \
			--expression="s|^TZID:$originaltz.*$|TZID:$mappedtz|" \
			--expression="s|TZID=$originaltz[^:]*|TZID=$mappedtz|" \
			"$file"
	done
fi

[tpapastylianou]

@machinedgod / @EpicVoyage I had some trouble with your scripts, so I made my own. See #430 Leaving this comment here in case anyone bumps into this with the same problem. :)

_{@EpicVoyage your version in particular assumed all entries would appear as "TZID:" fields first, before they could appear as "TZID=" entries, and relied on that fact to detect and fix all the relevant timezones in the file; however, in my .ics file this assumption was not valid for some reason, which resulted in fixing most entries, but leaving "TZID=" entries without a corresponding "TZID:" one silently uncorrected (and thus showing the wrong time for the appointment on the calendar!)}

[waynew]

FWIW it could be sensible for calcurse to check either a dedicated file location e.g. $XDG_CONFIG_HOME/calcurse/tzmap or something to lookup tznames and/or allow an external application to map the tznames.