khal icon indicating copy to clipboard operation
khal copied to clipboard

Published 20 hours ago • verified publisher icon pimutils

Fail on some calendar isc.

Open sergejzr opened this issue 11 months ago • 4 comments
trafficstars

Bug Report

Describe the bug
Dear khal Developers, I am just about to start using khal for managing my event and encountered following issue when using Confluence calendar tool. When subscribing to an ICS calendar created by Confluence (but could be probably any other non-standard app), certain entries containing custom properties are causing Exception by khal, making the events unimportable. These custom properties include:

  • X-CONFLUENCE-CUSTOM-EVENT-TYPE
  • X-CONFLUENCE-CUSTOM-TYPE-ID
  • X-CONFLUENCE-CUSTOM-TYPE-TITLE

Example of a problematic feed:

BEGIN:VCALENDAR
PRODID:-//Atlassian Confluence//Calendar Plugin 1.0//EN
VERSION:2.0
CALSCALE:GREGORIAN
X-WR-CALNAME:Kontrolle Serverbetrieb
X-WR-CALDESC:
X-WR-TIMEZONE:Europe/Berlin
X-MIGRATED-FOR-USER-KEY:true
X-CONFLUENCE-CUSTOM-EVENT-TYPE;X-CONFLUENCE-CUSTOM-TYPE-ID=62da149e-0d91-
 4a92-bb21-981d554e6631;X-CONFLUENCE-CUSTOM-TYPE-TITLE=Team 4;X-CONFLUENC
 E-CUSTOM-TYPE-ICON=workshop;X-CONFLUENCE-CUSTOM-TYPE-REMINDER-DURATION=0
 :true
X-CONFLUENCE-CUSTOM-EVENT-TYPE;X-CONFLUENCE-CUSTOM-TYPE-ID=6cb80cae-c927-
 4b08-8910-3290e4bce2d2;X-CONFLUENCE-CUSTOM-TYPE-TITLE=Team 2;X-CONFLUENC
X-CONFLUENCE-CUSTOM-EVENT-TYPE;X-CONFLUENCE-CUSTOM-TYPE-ID=33353bcd-8b97-
 4044-aa2f-1e644448260a;X-CONFLUENCE-CUSTOM-TYPE-TITLE=Team 1;X-CONFLUENC
X-CONFLUENCE-CUSTOM-EVENT-TYPE;X-CONFLUENCE-CUSTOM-TYPE-ID=10ddcc87-0b5a-
 4674-82d4-12e3da10c539;X-CONFLUENCE-CUSTOM-TYPE-TITLE=Team 3;X-CONFLUENC
BEGIN:VEVENT
DTSTAMP:20241213T090926Z
DTSTART;VALUE=DATE:20230523
DTEND;VALUE=DATE:20230524
SUMMARY:Kontrolle: Michl S.
X-CONFLUENCE-CUSTOM-TYPE-ID:62da149e-0d91-4a92-bb21-981d554e6631
CATEGORIES:Team 4
SUBCALENDAR-ID:9ac40073-1af4-4a4e-a3dc-dbaa0ff1ff08
PARENT-CALENDAR-ID:ec83ce91-3dea-4d6b-b2a9-16fbe1a176a3
PARENT-CALENDAR-NAME:
SUBSCRIPTION-ID:
SUBCALENDAR-TZ-ID:Europe/Berlin
SUBCALENDAR-NAME:Kontrolle
EVENT-ID:83137
EVENT-ALLDAY:true
CUSTOM-EVENTTYPE-ID:62da149e-0d91-4a92-bb21-981d554e6631
DESCRIPTION:
ORGANIZER;X-CONFLUENCE-USER-KEY=035caa816bb85cf6016da5fe210a1b0f;CN=Flori
 an Lanzmich;CUTYPE=INDIVIDUAL:mailto:[email protected]
RRULE:FREQ=WEEKLY;INTERVAL=8;BYDAY=TU
CREATED:20230228T092931Z
LAST-MODIFIED:20230306T072151Z
SEQUENCE:2
X-CONFLUENCE-SUBCALENDAR-TYPE:custom
STATUS:CONFIRMED
UID:0161db22105621a0f717221f2f7cea1fae0e32ec9c783a96020167a88da06c0d
END:VEVENT
BEGIN:VEVENT
DTSTAMP:20241213T090926Z
DTSTART;VALUE=DATE:20230523
DTEND;VALUE=DATE:20230524
SUMMARY:Kontrolle: Michl S.
X-CONFLUENCE-CUSTOM-TYPE-ID:62da149e-0d91-4a92-bb21-981d554e6631
CATEGORIES:Team 4
SUBCALENDAR-ID:9ac40073-1af4-4a4e-a3dc-dbaa0ff1ff08
PARENT-CALENDAR-ID:ec83ce91-3dea-4d6b-b2a9-16fbe1a176a3
PARENT-CALENDAR-NAME:
SUBSCRIPTION-ID:
SUBCALENDAR-TZ-ID:Europe/Berlin
SUBCALENDAR-NAME:Kontrolle
EVENT-ID:83152
EVENT-ALLDAY:true
CUSTOM-EVENTTYPE-ID:62da149e-0d91-4a92-bb21-981d554e6631
DESCRIPTION:
ORGANIZER;X-CONFLUENCE-USER-KEY=035caa8177af42de0177bd3665bc0036;CN=Serge
 j Zerr;CUTYPE=INDIVIDUAL:mailto:[email protected]
RRULE:FREQ=WEEKLY;INTERVAL=8;BYDAY=TU
RECURRENCE-ID;VALUE=DATE:20230523
CREATED:20230228T094529Z
LAST-MODIFIED:20230306T072836Z
SEQUENCE:2
X-CONFLUENCE-SUBCALENDAR-TYPE:custom
STATUS:CONFIRMED
UID:0161db22105621a0f717221f2f7cea1fae0e32ec9c783a96020167a88da06c0d
END:VEVENT
END:VCALENDAR

If applicable: Stack Trace

khal list --include-calendar confluencetest
Unknown exception happened.
Traceback (most recent call last):
  File "/home/szerr/anaconda3/lib/python3.9/site-packages/icalendar/parser.py", line 238, in from_ical
    key, val = q_split(param, '=', maxsplit=1)
ValueError: not enough values to unpack (expected 2, got 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/szerr/anaconda3/lib/python3.9/site-packages/icalendar/parser.py", line 343, in parts
    params = Parameters.from_ical(st[name_split + 1: value_split],
  File "/home/szerr/anaconda3/lib/python3.9/site-packages/icalendar/parser.py", line 262, in from_ical
    raise ValueError('%r is not a valid parameter string: %s'
ValueError: 'X-CONFLUENC' is not a valid parameter string: not enough values to unpack (expected 2, got 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/szerr/anaconda3/lib/python3.9/site-packages/khal/khalendar/khalendar.py", line 398, in _update_vevent
    update(event.raw, href=href, etag=etag, calendar=calendar)
  File "/home/szerr/anaconda3/lib/python3.9/site-packages/khal/khalendar/backend.py", line 224, in update
    ical = cal_from_ics(vevent_str)
  File "/home/szerr/anaconda3/lib/python3.9/site-packages/khal/icalendar.py", line 539, in cal_from_ics
    cal = icalendar.Calendar.from_ical(ics)
  File "/home/szerr/anaconda3/lib/python3.9/site-packages/icalendar/cal.py", line 330, in from_ical
    name, params, vals = line.parts()
  File "/home/szerr/anaconda3/lib/python3.9/site-packages/icalendar/parser.py", line 352, in parts
    raise ValueError(
ValueError: Content line could not be parsed into parts: 'X-CONFLUENCE-CUSTOM-EVENT-TYPE;X-CONFLUENCE-CUSTOM-TYPE-ID=6cb80cae-c927-4b08-8910-3290e4bce2d2;X-CONFLUENCE-CUSTOM-TYPE-TITLE=Team 2;X-CONFLUENC': 'X-CONFLUENC' is not a valid parameter string: not enough values to unpack (expected 2, got 1)
warning: Skipping confluencetest/confluencetest.ics: Content line could not be parsed into parts: 'X-CONFLUENCE-CUSTOM-EVENT-TYPE;X-CONFLUENCE-CUSTOM-TYPE-ID=6cb80cae-c927-4b08-8910-3290e4bce2d2;X-CONFLUENCE-CUSTOM-TYPE-TITLE=Team 2;X-CONFLUENC': 'X-CONFLUENC' is not a valid parameter string: not enough values to unpack (expected 2, got 1)
warning: This event will not be available in khal.

To Reproduce

  1. Subscribe to an ICS calendar generated by Confluence.
  2. Ensure the calendar includes entries with X-CONFLUENCE-* custom properties.
  3. khal list

Expected behavior

  • ignore unknown attributes

Screenshots

OS, version, khal version and how you installed it:

  • The output of khal --version: khal, version 0.11.2.dev20+g0c47162.d20230530
  • Installation method: pip install khal
  • Python version: Python 3.9.12
  • OS: Xubuntu
  • Khal config file (dont think is any special, but for sake of completeness):
[calendars]

[[fdm]]
path = ~/.cleancalendars/fdm/


[[bdienst]]
path = ~/.cleancalendars/bdienst/



[[abwesenheit]]
path = ~/.cleancalendars/abwesenheit/




[locale]
dateformat = %Y-%m-%d
  • Versions of other Python packages: Output of pip freeze available upon request.

Additional context
This issue arises with ICS calendar subscriptions from Confluence. The custom properties are probably critical for Confluence-specific functionality but should not prevent khal from importing or displaying the corresponding events.

Suggested resolution:

  • Ensure khal handles unsupported custom properties gracefully, either by ignoring them or providing a warning about their presence.

sergejzr avatar Dec 20 '24 12:12 sergejzr

The first of the 4 of those entries appears to be complete and seems to be handled fine. The other 3 are truncated, which is where the problem lies.

d7415 avatar Dec 20 '24 13:12 d7415

Would it make sense to make khal more fault tolerant / ignorant in such cases? Unfortunately this is the original feed that confluence returns. At the moment I do pre-filtering with another script. Theoretically a filter could turn feeds to standard ical format before importing to khal. Maybe it is already possible and I overlooked something in documentation...

sergejzr avatar Dec 23 '24 08:12 sergejzr

There are two problems with looking at khal for this:

  • The error is raised in icalendar, upstream of khal. We may be able to handle it but it's more complicated and there's still a question of where to draw the line - Some users may be confused if we tell them everything is fine but they later find out several of their events were silently corrupted and data lost.
  • Confluence seems to be breaking its own files, so it would probably be worth raising with them first. It handles the first property fine, so it definitely looks like a bug rather than deliberate behaviour.

d7415 avatar Dec 23 '24 18:12 d7415

I think it would make sense to show a warning that some files are invalid and events won't be shown.

But I'm also unsure of how to best implement this.

WhyNotHugo avatar Dec 23 '24 21:12 WhyNotHugo