pimutils
error: Unknown error occurred for calendars
Hi,
I receive following error since today. Let me know if there is any more information I can provide.
- Your vdirsyncer version
[~]$ vdirsyncer --version
vdirsyncer, version 0.16.9.dev0+gb5dd092.d20200612
- If applicable, which server software (and which version) you're using Fastmail
- Your Python version
[~]$ python --version
Python 3.8.3
- Your operating system
Arch Linux
- Your config file
[~]$ cat .config/vdirsyncer/config
[general]
status_path = "~/.vdirsyncer/status"
[pair contacts]
a = "contacts_local"
b = "contacts_remote"
collections = ["from a", "from b"]
[pair calendars]
a = "calendars_local"
b = "calendars_remote"
collections = ["from a", "from b"]
[storage contacts_local]
type = "filesystem"
path = "~/.contacts"
fileext = ".vcf"
[storage contacts_remote]
type = "carddav"
url = "https://carddav.fastmail.com/"
username = "..."
password.fetch = ["command", "/home/radek/bin/getpw.py", "...", "..."]
[storage calendars_local]
type = "filesystem"
path = "~/.calendars"
fileext = ".ics"
[storage calendars_remote]
type = "caldav"
url = "https://caldav.fastmail.com/"
username = "..."
password.fetch = ["command", "/home/radek/bin/getpw.py", "...", "..."]
- Use
vdirsyncer -vdebugfor debug output. The output is sensitive, but please attach at least the last few lines before the error (if applicable), censored as necessary. This is almost always the most useful information.
error: Unknown error occurred for calendars/...: /dav/calendars/user/.../.../...
error: Use `-vdebug` to see the full traceback.
debug: File "/usr/lib/python3.8/site-packages/vdirsyncer/cli/tasks.py", line 62, in sync_collection
debug: sync.sync(
debug: File "/usr/lib/python3.8/site-packages/vdirsyncer/sync/__init__.py", line 135, in sync
debug: b_nonempty = b_info.prepare_new_status()
debug: File "/usr/lib/python3.8/site-packages/vdirsyncer/sync/__init__.py", line 57, in prepare_new_status
debug: for href, item, etag in (self.storage.get_multi(prefetch)
debug: File "/usr/lib/python3.8/site-packages/vdirsyncer/storage/dav.py", line 503, in get_multi
debug: raise exceptions.NotFoundError(href)
error: 1 out of 13 tasks failed.
I see the same issue. Unfortunately the calendars in question are not synchronized at all, after starting from scratch with empty status and empty local storage, nothing is synced anymore.
Oh, and by the way I'm also syncing with fastmail, so it is possible that this is an error on their side. @WhyNotHugo, is there a simple way to check the consistency of the caldav data? At least one of the calendars where this error happens does not have very personal data in it, so I would also be willing to share it.
(Your config looks fine, BTW, the fastmail-specific part is almost identical to mine).
Do you have event entries and todo entries in the same local calendar?
I don't, and the error also happens when I remove the status directory and start from scratch.
I have the issue with vdirsyncer 0.16.7 when synchronizing an iCloud account.
Five calendar, four synchronize properly but in one I get the same issue.
It seems like it cannot find one of the .ics files that (before) got from the given calendar? (follows the error part from -vdebug; the rest of the sync seems to go smooth).
debug: Already normalized: '/268437010/calendars/E71734DA-CC16-4DE5-AA5D-4B7BC03A66CC/F9905C5E-7612-4FA9-9D24-804232303ACB.ics'
debug: Already normalized: '/268437010/calendars/E71734DA-CC16-4DE5-AA5D-4B7BC03A66CC/6445043D-A199-490E-9317-5AD03DFFFCD2.ics'
debug: File "/usr/lib/python3/dist-packages/vdirsyncer/cli/tasks.py", line 64, in sync_collection
debug: sync.sync(
debug: File "/usr/lib/python3/dist-packages/vdirsyncer/sync/__init__.py", line 136, in sync
debug: b_nonempty = b_info.prepare_new_status()
debug: File "/usr/lib/python3/dist-packages/vdirsyncer/sync/__init__.py", line 58, in prepare_new_status
debug: for href, item, etag in (self.storage.get_multi(prefetch)
debug: File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 505, in get_multi
debug: raise exceptions.NotFoundError(href)
Syncing calendars/EEF9E72D-C708-4687-A29D-1885F7016143
debug: PROPFIND https://p24-caldav.icloud.com:443/268437010/calendars/EEF9E72D-C708-4687-A29D-1885F7016143/
debug: {'User-Agent': 'vdirsyncer/0.16.7', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: <?xml version="1.0" encoding="utf-8" ?>
debug: <D:propfind xmlns:D="DAV:">
debug: <D:prop>
debug: <D:resourcetype/>
debug: <D:getcontenttype/>
debug: <D:getetag/>
debug: </D:prop>
debug: </D:propfind>
debug:
debug: Sending request...
debug: 207
Given that the local configuration seem to be only calendar related, it is not obvious how to solve this issue. Maybe an exception catcher and convert a failing .ics reading into a warning, so the rest of the calendar can be retrieved? As currently no data is obtained for the whole calendar.
may try moving ~/.calenders and ~./.vdirsyncer away from home (so you can restore them later if needed) and regenerate the directories with: vdirsyncer discover vdirsyncer metasync vdirsyncer sync
It helped me out when I got a weird errors.
Thanks, @LeSnake04 , but that's what I meant above by "starting from scratch with empty status and empty local storage". It doesn't help. Either there is some bug in vdirsyncer or in the fastmail server implementation, I cannot really tell which is the case.
I have the same issue with vdirsyncer 0.16.7
debug: Normalized URL from '/caldav/Y2FsOi8vMC8zMQ/20120830T181649Z-1346422920000%40fe80%3A0%3A0%3A0%3A215%3A17ff%3Afedc%3A562c%15.ics' to '/caldav/Y2FsOi8vMC8zMQ/20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%15.ics'
debug: Skipping '/caldav/Y2FsOi8vMC8zMQ/20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%15.ics', properties are missing.
error: Unknown error occured for mailboxdotorg_calendars: /caldav/Y2FsOi8vMC8zMQ/20130125T144935Z-1359137220000@fe80:0:0:0:215:17ff:fedc:5224%15.ics
error: Use `-vdebug` to see the full traceback.
debug: File "/usr/lib/python3.8/site-packages/vdirsyncer/cli/tasks.py", line 64, in sync_collection
debug: sync.sync(
debug: File "/usr/lib/python3.8/site-packages/vdirsyncer/sync/__init__.py", line 136, in sync
debug: b_nonempty = b_info.prepare_new_status()
debug: File "/usr/lib/python3.8/site-packages/vdirsyncer/sync/__init__.py", line 58, in prepare_new_status
debug: for href, item, etag in (self.storage.get_multi(prefetch)
debug: File "/usr/lib/python3.8/site-packages/vdirsyncer/storage/dav.py", line 505, in get_multi
debug: raise exceptions.NotFoundError(href)
I have tried to start from scratch but the problem still persists.
Not a pretty solution, but just commenting out the lines that raise the exception worked for me.
I got the same error as @edigiacomo . I have multiple google calendars but only one of them yields this error. @ArmaanB Which lines did you edit? May you provide more details about your solution?
@ArmaanB Which lines did you edit? May you provide more details about your solution?
Sorry, it’s been a few years, I don’t recall what I did
@dennis-hamann I believe the fix is just to go to line 505 in dav.py (line may be different in your version, just whatever the error message says) and replace the raise exceptions.NotFoundError(href) with something like dav_logger.warning("Not found: {}".format(href)). That being said, I also had this issue in the past and I think it was fixed in some more recent version. So maybe updating is all you need...
@edigiacomo The URL for your calendar entry here have some pretty unusual character combinations, and I suspect that it's not being escaped properly somehow.
Can you use davcli and check its output? In particular davcli caldav ... list-calendar-components and davcli caldav ... get.
davcli is a small tool to inspect caldav remote servers, and I want to compare its output to what vdirsyncer is logging.
I have a similar issue and I notice that only happens with
start_date = "datetime.now() - timedelta(days=60)"
end_date = "datetime.now() + timedelta(days=60)"
I change from 150 to 60 and same issue, after remove it completely the issue gone, I guess is related with recurrent events that are too old so the next occurrence event Not Found the parent.
Also with start_date end_date the warning warning: Skipping identical href: appears
@edigiacomo The URL for your calendar entry here have some pretty unusual character combinations, and I suspect that it's not being escaped properly somehow.
Can you use
davcliand check its output? In particulardavcli caldav ... list-calendar-componentsanddavcli caldav ... get.
davcliis a small tool to inspect caldav remote servers, and I want to compare its output to what vdirsyncer is logging.
In list-calendar-components.txt you can find the output of list-calendar-components:
$ DAVCLI_PASSWORD="$PASSWORD" target/release/davcli caldav --server-url "$SERVER_URL" --username "$USERNAME" list-calendar-components /caldav/Y2FsOi8vMC8zMQ/ > list-calendar-components.txt
I have found the resource href of the calendar entry and it seems that it's escaped in the same way:
$ grep 20120830T181649Z list-calendar-components.txt
/caldav/Y2FsOi8vMC8zMQ/20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%15.ics
But I can't download the entry
$ DAVCLI_PASSWORD="$PASSWORD" target/release/davcli caldav --server-url "$SERVER_URL" --username "$USERNAME" get "/caldav/Y2FsOi8vMC8zMQ/20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%15.ics"
Error: http request returned 400 Bad Request
And I get the same error with every href in the entries list: maybe I'm using davcli wrong?
I don't know if it can be useful, but I managed to download the entry escaping the % (%25) in the string /caldav/Y2FsOi8vMC8zMQ/20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%15.ics (I have redacted some text to ***):
$ curl -u "$USERNAME:$PASSWORD" "${SERVER_URL}/caldav/Y2FsOi8vMC8zMQ/20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%2515.ics"
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Open-Xchange//7.10.6-Rev49//EN
BEGIN:VTIMEZONE
TZID:Europe/Rome
TZURL:http://tzurl.org/zoneinfo-outlook/Europe/Rome
X-LIC-LOCATION:Europe/Rome
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:19700101T000000Z
ATTENDEE;PARTSTAT=NEEDS-ACTION;CUTYPE=INDIVIDUAL;EMAIL=****@***.com:mailto:***@****.com
ATTENDEE;CN=****;PARTSTAT=ACCEPTED;CUTYPE=INDIVIDUAL;EMAIL=***@***.org:mailto:***@***.org
CLASS:PUBLIC
CREATED:20171106T161958Z
DESCRIPTION:****
DTEND;TZID=Europe/Rome:20120831T193500
DTSTART;TZID=Europe/Rome:20120831T162200
LAST-MODIFIED:20171106T161958Z
LOCATION:Bologna Centrale
ORGANIZER;[email protected]:mailto:[email protected]
SEQUENCE:0
SUMMARY:***
TRANSP:OPAQUE
UID:20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%15
END:VEVENT
END:VCALENDAR
Thanks for testing this out with davcli. This is some useful information! :)
For the record, the output of davcli is un-escaped. I.e.: the following is the un-escaped path:
/caldav/Y2FsOi8vMC8zMQ/20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%15.ics
You shouldn't need to escape the value that you provide for the get command -- there's some code that explicitly handles escaping it. Apparently you've found a bug.
What's the output of git rev-parse HEAD? I want to make sure that you're not trying an old version before I start further debugging this.
What's the output of
git rev-parse HEAD? I want to make sure that you're not trying an old version before I start further debugging this.
$ git rev-parse HEAD
0ef2830c23ff61b074f4ce0de8f04810e1dc4d1c
@edigiacomo It turns out the bug was in davcli (the command line interface itself), not in libdav (the actual caldav implementation).
Indeed, URLs with % work fine with the write. If you want to re-do the above tests with the latest version davcli, they should work.
Thanks @WhyNotHugo, I confim that the bug is fixed and I was able to run the tests:
$ git rev-parse HEAD
5b416e756b1a950727c754d7be7ba4de1d9047e9
$ target/release/davcli --version
davcli libdav-0.1.1-261-g5b416e7
$ DAVCLI_PASSWORD="$PASSWORD" target/release/davcli caldav --server-url "$SERVER_URL" --username "$USERNAME" list-calendar-components /caldav/Y2FsOi8vMC8zMQ/ > list-calendar-components.txt
$ grep 20120830T181649Z list-calendar-components.txt
/caldav/Y2FsOi8vMC8zMQ/20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%15.ics
# Note: some info are redacted to ***
$ DAVCLI_PASSWORD="$PASSWORD" target/release/davcli caldav --server-url "$SERVER_URL" --username "$USERNAME" get '/caldav/Y2FsOi8vMC8zMQ/20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%15.ics'
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Open-Xchange//7.10.6-Rev52//EN
BEGIN:VTIMEZONE
TZID:Europe/Rome
TZURL:http://tzurl.org/zoneinfo-outlook/Europe/Rome
X-LIC-LOCATION:Europe/Rome
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:19700101T000000Z
ATTENDEE;PARTSTAT=NEEDS-ACTION;CUTYPE=INDIVIDUAL;EMAIL=***@***:mailto:***@***
ATTENDEE;CN=Emanuele Di Giacomo;PARTSTAT=ACCEPTED;CUTYPE=INDIVIDUAL;EMAIL=***@***:mailto:***@***
CLASS:PUBLIC
CREATED:20171106T161958Z
DESCRIPTION:***
DTEND;TZID=Europe/Rome:20120831T193500
DTSTART;TZID=Europe/Rome:20120831T162200
LAST-MODIFIED:20171106T161958Z
LOCATION:Bologna Centrale
ORGANIZER;[email protected]:mailto:[email protected]
SEQUENCE:0
SUMMARY:***
TRANSP:OPAQUE
UID:20120830T181649Z-1346422920000@fe80:0:0:0:215:17ff:fedc:562c%15
END:VEVENT
END:VCALENDAR
@dennis-hamann I believe the fix is just to go to line 505 in
dav.py(line may be different in your version, just whatever the error message says) and replace theraise exceptions.NotFoundError(href)with something likedav_logger.warning("Not found: {}".format(href)). That being said, I also had this issue in the past and I think it was fixed in some more recent version. So maybe updating is all you need...
This solved the same problem with google calendar
