core icon indicating copy to clipboard operation
core copied to clipboard

Remote Calendar cannot parse Office365 calendar with invalid timezones

Open N1co93 opened this issue 8 months ago • 28 comments

The problem

Sadly I can't add my company Office365 calendar via ICS file into Home Assistant. I get the error "Invalid .ics file" See error message below. I anonymized my ICS URL.

The error message mentioned some wrong timezone, but the timezone seems to be set correctly. Sorry for my German screenshot

Image

I can import the ICS file from my Personal Outlook account without any problem, but the URL is different. Company O365: https://outlook.office365.com/owa/calendar/XXXX/XXXX/calendar.ics Private Outlook: https://outlook.live.com/owa/calendar/XXXX/XXXX/XXXX/calendar.ics

Are there any additional information I can provide?

What version of Home Assistant Core has the issue?

core-2025.4.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Remote Calendar

Link to integration documentation on our website

https://www.home-assistant.io/integrations/remote_calendar

Diagnostics information

No response

Example YAML snippet


Anything in the logs that might be useful for us?

2025-04-03 09:25:57.046 DEBUG (MainThread) [homeassistant.components.remote_calendar.config_flow] User input: {'calendar_name': 'arbeit2', 'url': 'https://outlook.office365.com/owa/calendar/XXXX/XXXX/calendar.ics'}
2025-04-03 09:26:01.597 DEBUG (MainThread) [homeassistant.components.remote_calendar.config_flow] Invalid .ics file: Failed to parse component: 1 validation error for Event
__root__
  IcsTimezoneInfo.__init__() missing 1 required positional argument: 'timezone' (type=type_error)

Additional information

No response

N1co93 avatar Apr 03 '25 08:04 N1co93

Hey there @thomas55555, mind taking a look at this issue as it has been labeled with an integration (remote_calendar) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of remote_calendar can trigger bot actions by commenting:

  • @home-assistant close Closes the issue.
  • @home-assistant rename Awesome new title Renames the issue.
  • @home-assistant reopen Reopen the issue.
  • @home-assistant unassign remote_calendar Removes the current integration label and assignees on the issue, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)


remote_calendar documentation remote_calendar source (message by IssueLinks)

home-assistant[bot] avatar Apr 03 '25 08:04 home-assistant[bot]

IcsTimezoneInfo.__init__() missing 1 required positional argument: 'timezone' (type=type_error)

https://github.com/allenporter/ical/blob/9511bd591622640a617534d86a50e8323051a162/ical/timezone.py#L258 https://github.com/home-assistant/core/blob/8b3a43258dd673ae4db714e983e253d92eff0667/homeassistant/components/remote_calendar/config_flow.py#L70

IcsCalendarStream.calendar_from_ics

CloCkWeRX avatar Apr 03 '25 09:04 CloCkWeRX

Hard one to debug, as the backtrace is lopped off; unfortunately

CloCkWeRX avatar Apr 03 '25 09:04 CloCkWeRX

I can confirm this problem for my O365 work calendar, too. Maybe it is because of the @ in the url, or the really long url? I won't paste the original URL, but the scheme it follows: https://outlook.office365.com/owa/calendar/[email protected]/52chars/49chars/reachcalendar.ics Downloading the ics via wget or any browser works flawlessly.

The debug log shows that it already parsed some events succesfully and than ends with

2025-04-03 20:16:51.419 DEBUG (SyncWorker_13) [ical.component] Using 'priority' bare property value '5'
2025-04-03 20:16:51.419 DEBUG (SyncWorker_13) [ical.component] Parsing field 'rrule' with value 'FREQ=WEEKLY;UNTIL=20250319T110000Z;INTERVAL=1;BYDAY=MO,TU,WE,TH,FR;WKST=MO' as types [<class 'ical.types.recur.Recur'>]
2025-04-03 20:16:51.419 DEBUG (SyncWorker_13) [ical.component] Decoding 'rrule' as type '<class 'ical.types.recur.Recur'>'
2025-04-03 20:16:51.419 DEBUG (SyncWorker_13) [ical.types.date_time] DateTimeEncoder returned 2025-03-19 11:00:00+00:00
2025-04-03 20:16:51.420 DEBUG (MainThread) [homeassistant.components.remote_calendar.config_flow] Invalid .ics file: Failed to parse component: 1 validation error for Event
__root__
  IcsTimezoneInfo.__init__() missing 1 required positional argument: 'timezone' (type=type_error)

norgoe avatar Apr 03 '25 18:04 norgoe

It's also a problem for my O365 calendar. I tried encoding the @ in the URL as %40. Still didn't work.

rywalters avatar Apr 03 '25 20:04 rywalters

Based on a forum post about a related O365 calendar issue, I wonder if setting the timezone via the following powershell command would help. There's a bunch of work required to get remote powershell working for Exchange Online, so I haven't tried it yet.

Set-MailboxRegionalConfiguration -Identity {calendar's identity} -TimeZone "Pacific Standard Time"

rywalters avatar Apr 03 '25 20:04 rywalters

I downloaded the ics and uploaded it to my nextcloud server and created a share link and tried to import that - same error. Thus the @ in the url is not the problem. Seems to be the ical parsing. Maybe a first workaround would be to not stop loading the ics if one appointment fails because of timezone? Nextcloud btw imports the ics from the original location. IOS and MacOS can import or subscribe it from o365, too.

norgoe avatar Apr 03 '25 20:04 norgoe

It's not a problem only with O365, same problem happens on Apple calendars and google calendars

So I think this is a duplicated of this same issue that was created a few hours earlier: https://github.com/home-assistant/core/issues/142122

Since this issue is a duplicate, should be closed. And we should focus all the efforts in one issue.

edurenye avatar Apr 03 '25 23:04 edurenye

I checked the file with https://icalendar.org/validator.html and get a lot of errors like this:

Invalid TZID value or missing VTIMEZONE component (Central Europe Standard Time) near line # 57Reference: 3.2.19. Time Zone Identifier

jd1 avatar Apr 07 '25 15:04 jd1

Hello,

As JD1 says, it still not working because of this TZID value.

Here is what I got in the log : Error reading the calendar information: Failed to parse calendar EVENT component: Failed to validate: 20240413T220000, errors: (["Expected DATE-TIME TZID value 'W. Europe Standard Time' to be valid timezone", "Expected value to match DATE pattern: '20240413T220000'"])

ptitboogy avatar Apr 14 '25 12:04 ptitboogy

Same problem here.

Error reading the calendar information: Failed to parse calendar EVENT component: Failed to validate: 20240731T150000, errors: (["Expected DATE-TIME TZID value 'Central Europe Standard Time' to be valid timezone", "Expected value to match DATE pattern: '20240731T150000'"])

cypherbits avatar Apr 19 '25 21:04 cypherbits

Seems like outlook doesn't use standard timezones like they are defined: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones Do this is the reason why the data can't be parsed.

Thomas55555 avatar Apr 19 '25 21:04 Thomas55555

The issue is that Office365 is producing ics files that is not compliant with the RFC.

For details see the issue I raised in the actual library repo: https://github.com/allenporter/ical/issues/468#issuecomment-2819898569

Given the issues with non compliant ICS content (see other issues), a switch to a more forgiving library may be in order. The "ics" library is more forgiving and have no problem parsing my faulty Office365 ones.

jonnybergdahl avatar Apr 23 '25 18:04 jonnybergdahl

Until the parser is more forgiving, I am using https://github.com/franc6/ics_calendar although @franc6 has stated it's project will likely eventually close too.

sjoerdvanderhoorn avatar Apr 23 '25 18:04 sjoerdvanderhoorn

If I remember correctly what I saw when I took a quick peek at the component, it's using icalendar directly to parse the ICS files. I'd recommend using recurring-ical-events, instead. It's a wrapper for icalendar that provides additional functionality, too. I'd also recommend checking for and removing any NULL characters from the ics file -- some common calendaring systems (MS Exchage is one) incorrectly embed those, particularly in TZIDs.

franc6 avatar Apr 23 '25 20:04 franc6

Given the issues with non compliant ICS content (see other issues), a switch to a more forgiving library may be in order. The "ics" library is more forgiving and have no problem parsing my faulty Office365 ones.

If we switch this integration to another library it won't support other parts of rfc5545 correctly. We can patch remote_calendar or we can convince me to hack up ical but i'd rather hack up remote_calendar.

allenporter avatar May 04 '25 15:05 allenporter

One other thought is: I have a bunch of these rules in gcal_sync for fixing google calendar bugs. Maybe we can make a compat modules in ical that includes some of this stuff off to the side without mucking up the core library. https://github.com/allenporter/gcal_sync/blob/0b8be850938b12067a61f04989c64ef33d165ceb/gcal_sync/model.py#L665

That way remote_calendar can use it to fix all the broken calendars out there without allowing all usage of ical to silently violate the spec. For example, I don't think local_calendar should be allowed to silently make corrupt calendars.

allenporter avatar May 04 '25 15:05 allenporter

Related discussion: https://github.com/home-assistant/core/issues/142122#issuecomment-2801778588

allenporter avatar May 04 '25 15:05 allenporter

It's not a problem only with O365, same problem happens on Apple calendars and google calendars

So I think this is a duplicated of this same issue that was created a few hours earlier: #142122

Since this issue is a duplicate, should be closed. And we should focus all the efforts in one issue.

These are not duplicates, they are different root issues for each provider that need separate fixes.

allenporter avatar May 04 '25 20:05 allenporter

I've only seen one example of an .ics file hitting this bug which was when TZID is set to Customized Time Zone but VTIMEZONE is set to W. Europe Standard Time.

In other cases, we see TZID values but not the VTIMEZONE from the calendar. Can someone else share?

allenporter avatar May 05 '25 00:05 allenporter

TZID values are supposed to be represented on the parsing system OR be defined in a VTIMEZONE. One of the problems with ICS files from Office365 is the TZID imay be set to a time zone recognized by MS’s servers. In those cases, Office365 doesn’t bother putting the time zone definition in the exported ICS. When I was tracking down the cause of this problem, I found several references to people complaining about it including time zone definitions for everything (both for common time zones and for time zones that were not in use). I expect the current behavior is a result of those and other reports. Everybody is stuck with oiling the squeaky wheels, especially when the RFC leaves some items as optional (such as what time zones to include).

Another problem I saw, though, was embedded NULL characters in the TZIDs. These were always at the end of the actual ID and acted as padding. My guess is the producer was just dumping a buffer of a static size. Hence, my suggestion to remove any NULL characters, too. :)

franc6 avatar May 05 '25 01:05 franc6

That is not my understanding of reading the rfc5545. An rfc5545 calendar must be fully specified. It's clearly stated multiple times that it is required:

An individual "VTIMEZONE" calendar component MUST be specified for each unique "TZID" parameter value specified in the iCalendar object.

Still, I found another example that did have VTIMEZONE which ws correct, just not with the same timezone as in the TZID.

But just to be clear, i'm asking for example calendars for verification.

allenporter avatar May 05 '25 02:05 allenporter

This issue should be gone in HA 2025.5.0 , right? I still see the same error in my log with 2025.5.0:

2025-05-08 11:36:10.092 ERROR (MainThread) [homeassistant.components.remote_calendar.ics] Error parsing calendar information: Failed to parse calendar EVENT component: IcsTimezoneInfo.__init__() missing 1 required positional argument: 'timezone'

Should we reopen this issue?

N1co93 avatar May 08 '25 09:05 N1co93

Same here...

cypherbits avatar May 08 '25 09:05 cypherbits

I could add my Outlook calendar after updating to 2025.5.0 and I don't see any errors in the logs.

jd1 avatar May 08 '25 09:05 jd1

Mine still crashes:

calendar.zip

jakoss avatar May 08 '25 12:05 jakoss

Taking a look. Thanks for the reproduction case, i can reproduce it locally. It looks like the remaining issue is with recurring calendar event rules.

allenporter avatar May 10 '25 16:05 allenporter

I found a bug when doing a deepcopy on a timezone info object, so that can be fixed.

The other issue is that office 365 exports a calendar with a start date of year 1601. Very interesting choice that breaks some performance assumptions.

BEGIN:VTIMEZONE
TZID:Central European Standard Time
BEGIN:STANDARD
DTSTART:16010101T030000
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=-1SU;BYMONTH=10
END:STANDARD

allenporter avatar May 10 '25 17:05 allenporter

I just tried and got this error:

Error parsing calendar information: Failed to parse calendar EVENT component: IcsTimezoneInfo.init() missing 1 required positional argument: 'timezone'

I'm running 2025.5.1.

loryanstrant avatar May 16 '25 04:05 loryanstrant

Fixes should landin 2025.5.2

allenporter avatar May 17 '25 02:05 allenporter