Support non-floating appointments (respecting timezone changes)

[CrfzdPQM6]

Version 4.7.1

Bug description. Timezone information about events seems to be absent in calcurse functionality

Reproduce. Open calcurse in one timezone: env TZ=EST calcurse make an event: a ... 09:00 ... 10:00 ... My new event open calcurse in another timezone: env TZ=UTC calcurse` The event appears at 09:00, even though calcurse clearly understands the TZ variable because the local-time clock is different.

Expected Behavior. I would expect the event created at 09:00 when TZ=EST to appear at 09:00+4=13:00 when calcurse is opened with TZ=UTC

Is the creation timezone information: a) stored by calcurse? b) used during calcurse display? c) can it be set manually for particular events?

This is particularly important because events created in EST change follow a different daylight saving time pattern to those created in GMT, for example.

[lhca]

Calcurse creates, saves and displays time information of events (appointments) as broken-down time or local time, which is always relative to a time zone. See https://www.gnu.org/software/libc/manual/html_node/Time-Basics.html.

Hence, the time information of events is not bound to any particular time zone, e.g. the time zone valid when the event was created. A nine AM o'clock event is always displayed at 9:00 irrespective of the time zone.

[CrfzdPQM6]

Thanks for the fast reply! Are you saying that calcurse saves no timezone information along with events? This is hard to imagine because calcurse clearly has awareness of timezones as far as displaying the local time in the bottom left goes.

If calcurse always displays events in local time, what happens if the local time of my computer changes between the moment of event creation and a future moment when I might need to inspect the event? Surely in order to respect the intent to display events in local time, calcurse would need to know about the change in timezone? Otherwise it is no longer displaying the event in local time, after I move timezone. I'd need to start remembering what timezone I was in when I created the event, which is a recipe for confusion :)

If calcurse doesn't update event time to match local time, why does it update the clock to match local time? This seems to imply a fundamental disparity in timezone treatment, which ends up being quite confusing. Is this something we could develop?

[CrfzdPQM6]

This also seems problematic for daylight saving time where the event occurs at a fixed time in a country that doesn't align or doesn't respect european daylight saving time. Do I have to 'fix' all my repeating events?

[lhca]

How long have you used calcurse? Events are not created in particular time zones, but always relative to the time zone calcurse happens to be running in. Time zone information is handled by the operating system and is accessible to applications through standardized library functions.

The "problems" you describe, are handled by the underlying system. Calcurse does not save any time zone information for events, but only the local time, see https://www.gnu.org/software/libc/manual/html_node/Date-and-Time.html.

[CrfzdPQM6]

Thank you for your response, @lhca, I'm new to calcurse and I apologise in advance for any misunderstanding on my part! Does calcurse change anything about the display of events when the operating system time zone changes? I was expecting that changing the TZ variable (in my original post) would be enough to tell calcurse that it was running now in a new timezone, but no events moved.

[CrfzdPQM6]

Thanks, @lhca , I've read the gnudocs on Date-and-Time, in particular 21.5 (https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html). However, all that does is reaffirm for me that the TZ variable, that I tried, was the right way to inform, artificially, calcurse about a change in timezone.

So forgive me a really simple question. If I:

1. env TZ=UTC calcurse
2. Create an event at localtime 09:00
3. env TZ=EST+5 calcurse Why doesn't the event appear at localtime 04:00? The event appears at 09:00 which is an important loss of information and is only the 'local time of the event in the time zone of event creation', which requires me to remember which timezone I was in when I create each event in my calendar (let alone daylight saving, see below). That introduces enough uncertainty about the correct time of events that the calendar is made much less useful...

My use case, specifically, is that I want to be able to create an event that occurs at the same time of day, regardless of whether daylight saving time is in operation on the US East Coast, and then to have that event appear at the right localtime when my TZ is set to CET or CEST. That's non-trivial because the dst 'start' and 'end' (https://www.gnu.org/software/libc/manual/html_node/TZ-Variable.html) occur on different days in Europe and USA. calcurse should handle all of this behind the scenes, I think.

Thanks in advance for your patience in walking me through this.

[CrfzdPQM6]

I agree calcurse has no need to save any information about timezones, but instead of saving event information in the current localtime (dangerous, because localtime is ill-defined) it would be better to save event information in UTC (easily obtained from the value of $TZ) and then render the event information in the local timezone (as defined by $TZ).

There would then be no need to add timezone info to calcurse, but the rendering would be 'timezone-safe'

[CrfzdPQM6]

Interestingly this is exactly the same solution that @waynew described in https://github.com/lfos/calcurse/issues/329.

@lhca , you've given some very detailed (and patient!) explanations in https://github.com/lfos/calcurse/issues/323#issuecomment-718876253 and https://github.com/lfos/calcurse/issues/323#issuecomment-719908117 that are helpful on this topic, but they address the import of events in specific timezones as opposed to their display.

Is it possible to cause events to change their displayed localtime based on a TZ change on the system?

[CrfzdPQM6]

Just for anyone else who browses this thread later, I don't think calcurse has the required functionality. I recommend instead something like khal, which handles timezones like I'd expect:

> khal new 09:00 11:00 UTC "my new event" "and a description"
> khal list
Today, 29/03/2022
11:00-13:00 my new event and a description
> TZ=EST khal list
Today, 29/03/2022
04:00-06:00 my new event and a description

The calcurse UI is prettier, but you can't have everything I guess!

[lhca]

This is a case of unfulfilled expectations and not, I would argue, a deficiency in calcurse.

But what can a new user reasonably expect of calcurse with regard to the topic at hand (timezone information)? And is it documented anywhere? To the latter: not that I know of. To the former: I will use myself and my encounter with calcurse as a "use case".

I came across calcurse in May 2017, liked it and have used it ever since. Until that time, I had used a paper calendar for many years. I am an old man. Each year I bought a new one or got one from my employer. So what did I expect of calcurse? Nothing that I can remember! But I used it, much like I had used my paper calendar. And that's the point, I believe. Calcurse is much like a paper calendar. What you enter is local time without timezone information. Most of my paper-calendar entries were tied to daily activities and routines and were independent of timezones. My paper calendar worked just as well when I was in a foreign country where I had to adjust the clock. And so does calcurse. To take a silly example: an appointment saying "eight o'clock, breakfast" is tied to your day and has nothing to do with the timezone you are in. If that appointment were created with an absolute date-time value (including timezone), it would be as if your watch were never adjusted to the current timezone and always displayed the time of your home-timezone, and you insisted on having breakfast when it showed 8 o'oclock irrespective of where in the world you happened to be.

But that is just my own "use case" and surely not the only reasonable one. That calcurse must be aware of timezones and should save absolute date-time information are major requirements and imply a lot of thought and work. It would be interesting if you could supply a use case for this.

The iCalendar standard (RFC 5545) recognizes three (date-)time forms: (1)local time, (2)UTC time and (3)local time and timezone reference. The first form is also called floating time and is what calcurse uses for all appointments.

Only when you import an iCalendar event, does calcurse consider the timezone of the event, if given. The date-time of is converted to local time of the current timezone, and an appointment created with an attached note containing the timezone information from the imported event.

[CrfzdPQM6]

Thank you, @lhca , for taking the time to respond so thoughtfully.

Let me give three use cases in making the case for timezone awareness.

1. I work with colleagues in Europe/* America/New_York, Asia/Shanghai, and many arrange meetings in their own timezone. It would be great if I could enter the meeting time in the declared timezone (they don't send an iCalendar event) and let calcurse handle the conversion to my local timezone.

2. I frequently change timezone myself (+/- 1 hour) and it renders event-reminders nearly useless if my events don't update to my local timezone, as it's not feasible to remember where I was when I created the calcurse event. I suppose I could write my location at the moment of creation in the event description, but that's just making me do work that calcurse could do better ;)

3. I have a meeting which is regular in the "America/New_York" timezone, in which a "9 am" event happens at 9am regardless of daylight saving (i.e. its time wrt UTC varies). US and European daylight saving adjustments are not quite aligned (2 weeks) so it would have been great for a meeting defined at "9am America/New_York" (which would incorporate DST) to automatically shift by 1 hour when interpretted in my European timezone during those two weeks where the timezones are not aligned.

In all three cases, being timezone aware (both in "UTC+X" form, and in "Region/City" (to fix DST variations) form) would be important.

The distinction between 'unfulfilled expectation' and 'deficiency' depends on the observer, I think, and I can see how, for you, calcurse does just fine. But, for me, the absence of this fairly basic functionality renders calcurse unusable so to me it's definitely a deficiency.

[lfos]

Thank you both for the constructive discussion. I agree that there are valid use cases for both floating-time appointments and fixed-time appointments. If we were to add functionality for the latter to calcurse, here are two options:

1. From a UX perspective, appointments are either floating or fixed. This type is chosen when the appointment is created and can be modified later. When stored, for each appointment, a Boolean flag distinguishes whether stored time should be interpreted as UTC or as local time. For user interaction, times are always transformed to local time, and transformed back for storage.
2. For each appointment, time zone information can be attached. This information is added when an appointment is created and can be edited later. This information is optional, and if omitted, the date/time is treated as floating time. For display, times are always transformed to local time. (When editing, either local time or stored time + timezone can be used.)

Option (1) should be slightly easier to implement, option (2) might require more work in terms of user interface and backwards-compatible integration in our storage format.

Option (2) has the benefit of keeping more metadata and allowing iCal import/export to retain original time zone information.

Are there other benefits or challenges I am missing? Can either of you think of any other options that are worthwhile considering?

[CrfzdPQM6]

Hi @lfos , sounds smart. To maintain the current behaviour as far as possible, I'd make the explicit specification of a timezone the distinguishing factor: no tz = localtime (default); tz specified = absolute time (new feature).

I'm not an expert here, but it'd be great to have two nuances to tz treatments:

• accept "0900 EST" as a 'time'
• accept "0900 America/New_York" as a 'time'. This would be different to the above in its behaviour at a daylight-saving change, where the first treatment would cause the event to move by 1 hour (which may not be desirable e.g. weekly-meeting at 9am "New York time", regardless of daylight saving.

Thanks so much, everyone!

PS: Would be very happy to chip in. My familiarity with the codebase is small, so I can't guarantee much, but if there's something easy to start with feel free to suggest it.

[waynew]

One thing that I think has been touched on during this conversation but I think is worth spelling out explicitly is that time is ~an illusion~ hard.

What do I mean by that?

If I created a recurring event with the time America/Chicago, as is my local timezone (Central Time) lasting from February 1 to April 1, at 5:00PM, and I shared that event with someone in, say India...

In February, that would correspond to 4AM India time - see http://timesched.pocoo.org/?date=2022-02-01&tz=central-standard-time!,mountain-standard-time,india-standard-time&range=1020,1080

But today that's 3AM India time, see http://timesched.pocoo.org/?date=2022-04-06&tz=central-standard-time!,mountain-standard-time,india-standard-time&range=1020,1080

In the ical format, you can define timezones that look something like this:

BEGIN:VTIMEZONE
TZID:Mountain Standard Time
BEGIN:STANDARD
DTSTART:16010101T020000
TZOFFSETFROM:-0600
TZOFFSETTO:-0700
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=1SU;BYMONTH=11
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:16010101T020000
TZOFFSETFROM:-0700
TZOFFSETTO:-0600
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=2SU;BYMONTH=3
END:DAYLIGHT
END:VTIMEZONE

(I think it's also considered legal to reference America/Chicago, for instance)

So... times are complicated when you're dealing with people :joy:

I'm not sure if the plan would be to just keep mostly with the python stdlib or use a library. Unfortunately I know that there are actually a lot of bugs in certain time style libraries :disappointed:

Python has a tzinfo base class that would pretty naturally be able to use the above sort of definition. TZOFFSETFROM (and ...TO) are the UTC offsets, so I don't think it would be awful to compute the times

A first blush looks something like

[``` class DaylightStandard: def init(self, start, offset_from, offset_to, rule): self.start = start self.offset_from = offset_from self.offset_to = offset_to self.rule = rule

@property
def offset(self):
    '''
    The UTC offset this Daylight/Standard time represents.
    '''

def contains(self, datetime):
    '''
    Return True if the datetime falls within the DaylightStandards
    recurrence rule, False otherwise.
    '''
    # magic goes here - Python has lookups for which month/day things are,
    # and it should be possible to figure out intervals from there.

class Timezone(tzinfo): def init(self, ical): self.id = ical.tzid self.standard = DaylightStandard( start=ical.standard.dtstart, offset_from=ical.standard.offset_from, offset_to=ical.standard.offset_to, rule=ical.standard.rrule, ) self.daylight = DaylightStandard( start=ical.daylight.dtstart, offset_from=ical.daylight.offset_from, offset_to=ical.daylight.offset_to, rule=ical.daylight.rrule, )

def dst(self, dt):
    if self.standard.contains(dt):
        return self.standard.offset
    elif self.daylight.contains(dt):
        return self.daylight.offset
    else:
        raise TimeOutOfTimeError(f'{dt} does not fall within a standard or daylight time for this zone info')

def utcoffset(self, dt):
    ...

def tzname(self, dt):
    return self.id

That's not complete by any means, but hopefully it gives some flavor about the kind of weird complications that can arise when it comes to the time of things :sweat_smile: 

I'm more than happy to help review code if someone needs - though I'm typically pretty busy :upside_down_face: or I would have just written this all myself already :joy: 

[CrfzdPQM6]

One thing that I think has been touched on during this conversation but I think is worth spelling out explicitly is that time is ~an illusion~ hard.

What do I mean by that?

If I created a recurring event with the time America/Chicago, as is my local timezone (Central Time) lasting from February 1 to April 1, at 5:00PM, and I shared that event with someone in, say India...

In February, that would correspond to 4AM India time - see http://timesched.pocoo.org/?date=2022-02-01&tz=central-standard-time!,mountain-standard-time,india-standard-time&range=1020,1080

But today that's 3AM India time, see http://timesched.pocoo.org/?date=2022-04-06&tz=central-standard-time!,mountain-standard-time,india-standard-time&range=1020,1080

Thanks , @waynew , that's an exact parallel to the situation I hoped we'd tackle - events that have a constant localtime in another timezone with respect to mine, where that other timezone follows daylight saving on a different schedule to mine.