calcurse
calcurse copied to clipboard
lfos
calcurse-caldav: Corrupted sync DB when iCal file contains multiple events
I'am able to initialize the Database, but if i try to sync it, calcurse-caldav can't read the initialized database file. My Server is running Baikal 0.5.0
Connecting to ///SERVER///...
Loading synchronization database from /home/witti/.calcurse/caldav/sync.db...
Traceback (most recent call last):
File "/usr/bin/calcurse-caldav", line 519, in <module>
syncdb = get_syncdb(syncdbfn)
File "/usr/bin/calcurse-caldav", line 196, in get_syncdb
href, etag, objhash = line.rstrip().split(' ')
ValueError: not enough values to unpack (expected 3, got 1)
EDIT: It works, after editing sync.db manually. The init process made some dublicated values into new lines (the 'objhash' came up a second time in a line just beneath its element), which had to be deleted. So the problem isn't the synchronization itself but the initialization.
Just installed calcurse 4.2.2 initialized database for calcurse-caldav and am experiencing the same error ....Loading synchronization database from ~/user/.calcurse/caldav/sync.db...
Traceback (most recent call last):
File "/usr/local/bin/calcurse-caldav", line 519, in
Sounds like your synchronization database is defective. There is probably some line in ~/.calcurse/caldav/sync.db that contains no spaces (every line should consist of three tokens separated by spaces). Could you check and let me know what the offending line looks like? Maybe also have a look at the line before and after.
sync.db contained;
'/dav/calendars/user/[email protected]/788c1e89-2580-4556-8337-
b47ef7a89f64/[email protected]
ab813d7a9ac84cd7c90733c6e0bf620942cedaee
Some items could not be imported, see log file? [y/n]'
i removed the line and grepped for the associated entry. found the entry and then deleted it from the remote server and corresponding local entry. reran calcurse-caldav --init=keep-remote followed by calcurse-caldav. Same error resulted as I had originally posted. This time I found offending entries in sync.db, different from the previous time; 2 double entries and two of the same 'some items could not be imported..' messages;
'/dav/calendars/user/[email protected]/788c1e89-2580-4556-8337-b47ef7a89f64/2735d1ae3b9434a1095e99397b0609c6ee249200.ics 53691f70a24ca9413566bd3a610c312746831e2e
2735d1ae3b9434a1095e99397b0609c6ee249200
2735d1ae3b9434a1095e99397b0609c6ee249200'
'/dav/calendars/user/[email protected]/788c1e89-2580-4556-8337-b47ef7a89f64/f47a4ec0a5a05b2c91e30f2f25a28922259702f9.ics 00488be9f1262f6126651859fc822801d0778769
58dd2572925331bc1770ad5aabe32091a5b33070
58dd2572925331bc1770ad5aabe32091a5b33070'
'/dav/calendars/user/[email protected]/788c1e89-2580-4556-8337-b47ef7a89f64/[email protected] ab813d7a9ac84cd7c90733c6e0bf620942cedaee Some items could not be imported, see log file? [y/n]'
'/dav/calendars/user/[email protected]/788c1e89-2580-4556-8337-b47ef7a89f64/[email protected] f619a48032b2d8977cd76754f8ce9e3519ea643a Some items could not be imported, see log file? [y/n]'
resyncing the same caldav addresses using vdirsyncer/ikhal though currently returns no errors.
There are some improvements to the synchronization script in the pu branch. Maybe you can check that out and see whether it helps?
I think we found the culprit in the meantime: Such lines seem to originate from iCal resources with multiple events; see pull request #60.
I thought about this for a while and it seems there are a couple of issues.
Firstly, to support iCal streams with multiple events, we would have to extend the synchronization database format used within calcurse-caldav. The current data structure maps each remote href to a tuple. The tuple consists of an ETag (which is the ETag of the remote object at the time of the last synchronization) and a local object identifier (uniquely identifying the item within calcurse). The extension is pretty straightforward: Replace the local object identifier by a list of object identifiers.
However, there are some more things to be considered: Instead of requesting removal of a remote object when the hash is gone, we now have to update the iCal file and drop one of the items as it is removed. We must only remove the remote object if all its items are deleted locally.
Something that makes it even worse is the fact that calcurse does not keep track of appointments on updates. It only keeps track of the actual contents. Editing an appointment is equivalent to deleting the item and adding another item with the updated information. So, if we locally edit an item which is part of a multi-event iCal file, it looks like one of the events was removed and another one was added. With the current design, there is no way of telling whether the newly added item should be grouped with the events in the multi-event iCal file or whether it is an entirely separate/new item.
One possible and pretty simple solution is to make calcurse-caldav import multi-event iCal files and immediately convert them to multiple single-event iCal files (or, if we agree that pull synchronization should be a read-only operation, do the conversion when doing the push part of the follow-up synchronization). I am not sure whether it is a sensible thing to split up iCal files that way when syhncronizing, though. It should definitely be discussed.
On a related note: I just found [1] and I am a bit puzzled. Does anybody happen to know in which scenarios iCloud (or another CalDAV server) returns such "grouped" events?
[1] https://stackoverflow.com/questions/28551879/icloud-vcalendar-multiple-events
Hi @lfos, I have not created a special test-icloud calendar. Unfortunately I'm not able to reproduce this scenario with this one. I tried:
- multiple timed events on one day
- timed event spanned over multiple days
- full day events on one day
- full day events spanned over multiple days
- variations of the above
Creating those events in the iCloud Web-Interface does not lead to such grouped events. Maybe it's not iCloud but a calendar software publishing to it.
I'll play around with this a bit futher.
EDIT: I did the same test with the iOS Calendar App and Simple Calendar and aCalendar+ on Android syncronized via DavDroid. None are creating grouped events.
Regarding the event splitting. I'm for it. However, we should be careful and try hard to find out if there are negative consequences.
Looking again at the example you uploaded and linked to in the PR, it seems like the three "grouped" events actually belong to the same recurrent item and correspond to different occurrences. Maybe you can try to come up with an "exotic" recurrent appointment (weekly with some exceptions and some additional occurrences slightly off the regular interval) and see whether the effect is reproducible?
FTR, I've run into this issue with Google calendar, the problematic line in my sync.db being:
662f693bf58e73324bad533485126f40e9fb0199
So, it's not specific to iCal I guess
Found the offending event:
BEGIN:VCALENDAR
PRODID:-//Nextcloud calendar v1.6.1
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VEVENT
DTSTART:20131205T170000Z
DTEND:20131205T180000Z
DTSTAMP:20180412T150045Z
UID:[email protected]
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;CN=Kennet
h Flak;X-NUM-GUESTS=0:mailto:[email protected]
CREATED:20131204T135337Z
LAST-MODIFIED:20131204T135337Z
LOCATION:
SEQUENCE:1
STATUS:TENTATIVE
TRANSP:OPAQUE
CATEGORIES:http://schemas.google.com/g/2005#event
END:VEVENT
END:VCALENDAR
After deleting this from the server the caldav script works fine :-) Thanks for great software, by the way!
I too have the error with the Some items could not be imported, see log file? [y/n]-lines in the DB file. The current version in the pu branch just skips these events.
I'm having the same issue while importing recurrent events from Google Calendar that have been modified. It seems like, at least for Google, the grouping occurs when a recurring event changes on an individual date.
