vdirsyncer
vdirsyncer copied to clipboard
pimutils
vdirsyncer 0.19.0 rejects valid server certificate
After upgrading to vdirsyncer 0.19.0 (0.18.0 with the same dependency versions is confirmed to work), I cannot connect to any remote server, and vdirsyncer fails with a certificate error. Using requests at the Python REPL works.
My operating system is Debian Stable, but I am using it as a foreign distro for Guix; vdirsyncer is installed on Guix under Python 3.9.9.
This appears to affect all TLS servers; certainly it affects my own server, Google Calendar, and Microsoft-hosted Outlook calendars. An example sync config that fails is:
[pair ub_public_calendar]
a = "local_cal_cse_public"
b = "ub_cal_cse_public"
collections = null
[storage local_cal_cse_public]
type = "filesystem"
path = "~/.calendars/ub-public/"
fileext = ".ics"
[storage ub_cal_cse_public]
type = "http"
url = "https://calendar.google.com/calendar/ical/[redacted]%40group.calendar.google.com/public/basic.ics"
It yields this -vdebug log:
============
debug: GET https://calendar.google.com/calendar/ical/[redacted]%40group.calendar.google.com/public/basic.ics
debug: {'User-Agent': 'vdirsyncer/0.19.0'}
debug: None
debug: Sending request...
error: Unknown error occurred for ub_public_calendar: Cannot connect to host calendar.google.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')]
error: Use `-vdebug` to see the full traceback.
debug: File "/gnu/store/mz0sf0hai0byn5b43dv8cakl7vhq08yj-vdirsyncer-0.19.0/lib/python3.9/site-packages/vdirsyncer/cli/tasks.py", line 72, in sync_collection
debug: await sync.sync(
debug: File "/gnu/store/mz0sf0hai0byn5b43dv8cakl7vhq08yj-vdirsyncer-0.19.0/lib/python3.9/site-packages/vdirsyncer/sync/__init__.py", line 145, in sync
debug: b_nonempty = await b_info.prepare_new_status()
debug: File "/gnu/store/mz0sf0hai0byn5b43dv8cakl7vhq08yj-vdirsyncer-0.19.0/lib/python3.9/site-packages/vdirsyncer/sync/__init__.py", line 48, in prepare_new_status
debug: async for href, etag in self.storage.list():
debug: File "/gnu/store/mz0sf0hai0byn5b43dv8cakl7vhq08yj-vdirsyncer-0.19.0/lib/python3.9/site-packages/vdirsyncer/storage/http.py", line 73, in list
debug: r = await request(
debug: File "/gnu/store/mz0sf0hai0byn5b43dv8cakl7vhq08yj-vdirsyncer-0.19.0/lib/python3.9/site-packages/vdirsyncer/http.py", line 132, in request
debug: response = await session.request(method, url, **kwargs)
debug: File "/gnu/store/cx2ky0ksjig1gp5rlx47vfavgbddg6gw-python-aiohttp-3.8.1/lib/python3.9/site-packages/aiohttp/client.py", line 535, in _request
debug: conn = await self._connector.connect(
debug: File "/gnu/store/cx2ky0ksjig1gp5rlx47vfavgbddg6gw-python-aiohttp-3.8.1/lib/python3.9/site-packages/aiohttp/connector.py", line 542, in connect
debug: proto = await self._create_connection(req, traces, timeout)
debug: File "/gnu/store/cx2ky0ksjig1gp5rlx47vfavgbddg6gw-python-aiohttp-3.8.1/lib/python3.9/site-packages/aiohttp/connector.py", line 907, in _create_connection
debug: _, proto = await self._create_direct_connection(req, traces, timeout)
debug: File "/gnu/store/cx2ky0ksjig1gp5rlx47vfavgbddg6gw-python-aiohttp-3.8.1/lib/python3.9/site-packages/aiohttp/connector.py", line 1206, in _create_direct_connection
debug: raise last_exc
debug: File "/gnu/store/cx2ky0ksjig1gp5rlx47vfavgbddg6gw-python-aiohttp-3.8.1/lib/python3.9/site-packages/aiohttp/connector.py", line 1175, in _create_direct_connection
debug: transp, proto = await self._wrap_create_connection(
debug: File "/gnu/store/cx2ky0ksjig1gp5rlx47vfavgbddg6gw-python-aiohttp-3.8.1/lib/python3.9/site-packages/aiohttp/connector.py", line 988, in _wrap_create_connection
debug: raise ClientConnectorCertificateError(req.connection_key, exc) from exc
The vdirsyncer documentation says that this may be due to Python requests configuration due to requests shipping its own root CAs, but:
Python 3.9.9 (main, Jan 1 1970, 00:00:01)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.request(url="https://calendar.google.com/calendar/ical/rlqfvib3jur57f9vr9rq034nt4%40group.calendar.google.com/public/basic.ics", method="GET")
<Response [200]>
>>>
What's the output of which python3 (e.g.: the python version you run on a shell)?
What's the output of head $(which vdirsyncer) (e.g.: the python version used by vdirsyner)?
The python3 I run in the shell is not relevant here; when I was testing, I was using the python used by vdirsyncer, which is /gnu/store/slhyk2ng2nlxhz4bzqxavgsksbsgp9j6-python-wrapper-3.9.9/bin/python. (which python3 is /usr/bin/python3, to answer your question.)
The python used by vdirsyncer is above.
To be clear:
$ GUIX_PYTHONPATH="/gnu/store/nvj5yfa1ifc2w7awb47s8xnwdq676m1x-vdirsyncer-0.19.0/lib/python3.9/site-packages:/gnu/store/7ddk7p8zj5b4gihm0ri6d13fq27js43c-python-setuptools-scm-6.3.2/lib/python3.9/site-packages:/gnu/store/y0f66bycjr3n6yzk27i118a4xlrgb7ia-python-sphinx-5.1.1/lib/python3.9/site-packages:/gnu/store/dwlzb2aa1aw2z7629s4wk08pqmi49ail-python-aioresponses-0.7.2/lib/python3.9/site-packages:/gnu/store/7hcwk1acrih0wc49a15s60sb6w0c8gp2-python-hypothesis-6.0.2/lib/python3.9/site-packages:/gnu/store/avc0yawkvjwyc1nx93rfgza9xgwh7ncv-python-trustme-0.6.0/lib/python3.9/site-packages:/gnu/store/wnl4a5m90fqg3gaj7bfzwkk52lr0y84n-python-pytest-6.2.5/lib/python3.9/site-packages:/gnu/store/hm97w6qgapy8x7i341mhcdn7j3jxfb42-python-pytest-asyncio-0.17.2/lib/python3.9/site-packages:/gnu/store/4gz1xiqpb5nqhmb9w6hb4gd6rr18nrjl-python-pytest-cov-3.0.0/lib/python3.9/site-packages:/gnu/store/7jhmy0j4z50vb7051ckl0fkwv60y9aq8-python-pytest-httpserver-1.0.0/lib/python3.9/site-packages:/gnu/store/nhvngfip3gc20psqdakp63vrs1k9ym25-radicale-3.1.7/lib/python3.9/site-packages:/gnu/store/yz1cw1rl04n020hw7mkrq6z6vabgni3d-python-aiohttp-3.8.1/lib/python3.9/site-packages:/gnu/store/y12m9h6z8sp7gqwi07hr2j8872k86lhw-python-aiostream-0.4.5/lib/python3.9/site-packages:/gnu/store/1nnclhwqhb40j0s2yviy4mlzxy341m38-python-atomicwrites-1.4.0/lib/python3.9/site-packages:/gnu/store/c8kw84vmw1lz22j2fsfdjnmhaf7sh1ra-python-click-8.1.3/lib/python3.9/site-packages:/gnu/store/mx27xizxks1jj39ibdxik1mp12j7mziw-python-click-log-0.3.2/lib/python3.9/site-packages:/gnu/store/yl3am561sys3bqw27wjy68bg6lwdi9a1-python-requests-2.28.1/lib/python3.9/site-packages:/gnu/store/dpmx7c95kzr88jbycxvzjls75xghhfg5-python-requests-toolbelt-0.9.1/lib/python3.9/site-packages:/gnu/store/myxf1a4i84bm42301v7k990hq9zh316y-python-3.9.9/lib/python3.9/site-packages:/gnu/store/72vvhvcpia1jkwnnmp3862k94lak5410-python-tomli-2.0.0/lib/python3.9/site-packages:/gnu/store/jbb1l7nqy3dskqy8i835p8wbi871dmsy-python-packaging-bootstrap-21.3/lib/python3.9/site-packages:/gnu/store/24jkjxlbprrxw1nnbjsfla7qr3jii9sq-python-sphinxcontrib-serializinghtml-1.1.5/lib/python3.9/site-packages:/gnu/store/g0xrn4960b1jspvx5fa58paq3f9dzxbv-python-sphinxcontrib-qthelp-1.0.3/lib/python3.9/site-packages:/gnu/store/ky9dbvp70w8m6i43g6jbbafizbv8l6hh-python-sphinxcontrib-jsmath-1.0.1/lib/python3.9/site-packages:/gnu/store/8j9kdd52xh8idh2f7zf484vcx8cdc1bx-python-sphinxcontrib-htmlhelp-2.0.0/lib/python3.9/site-packages:/gnu/store/z63b58rr8c528q7dx0wqg0y99vb4sk20-python-sphinxcontrib-devhelp-1.0.2/lib/python3.9/site-packages:/gnu/store/4bp1wrhl4vds6l9pi84sri6nr3xas86q-python-sphinxcontrib-applehelp-1.0.2/lib/python3.9/site-packages:/gnu/store/cv0g6xambq6a869q06svzw968h833zyd-python-sphinx-alabaster-theme-0.7.12/lib/python3.9/site-packages:/gnu/store/8hbhvxj7jfmnqdy5wd5jv65hcr2dfxjr-python-snowballstemmer-2.0.0/lib/python3.9/site-packages:/gnu/store/swj9l6a19w3qvzb0339ycdsbvd55d2r1-python-pygments-2.12.0/lib/python3.9/site-packages:/gnu/store/kxlnph6hn0clsqbbf7b0f61hw51ak2g2-python-packaging-21.3/lib/python3.9/site-packages:/gnu/store/5v1f0kf5vgrajvl33fpf06y64dkyfbg0-python-importlib-metadata-4.11.3/lib/python3.9/site-packages:/gnu/store/k2nnrcyk8163k3rhapzx0lshfsccylbc-python-imagesize-1.2.0/lib/python3.9/site-packages:/gnu/store/zc8b68mc705c4q7ay33fjbzmkjm306bd-python-jinja2-3.1.1/lib/python3.9/site-packages:/gnu/store/00gpb1qff7wj6q8lm2av34kfs7yhq48m-python-docutils-0.17.1/lib/python3.9/site-packages:/gnu/store/p3r1mwdza7zz81nr36phx82b3b9iwc6a-python-babel-2.10.3/lib/python3.9/site-packages:/gnu/store/w7iby781rv62mdqdy3jx8njd7aq6zj7x-python-sortedcontainers-2.1.0/lib/python3.9/site-packages:/gnu/store/wp31hr5sia5wydha04ijqiz2kdhck4y0-python-attrs-bootstrap-21.2.0/lib/python3.9/site-packages:/gnu/store/yx6jr25a64xf5yf8w7d8qjp3s7p46c8y-python-cryptography-3.4.8/lib/python3.9/site-packages:/gnu/store/rh5pmm5ralyl06pgfr83qlfxaav6svjr-python-wcwidth-0.1.8/lib/python3.9/site-packages:/gnu/store/hmyqhci2vvrnvjwm32l26kwgasz7in1w-python-toml-0.10.2/lib/python3.9/site-packages:/gnu/store/xqvr1b5065idb5y8jxgr42cdkiwj6l64-python-six-bootstrap-1.16.0/lib/python3.9/site-packages:/gnu/store/ls2xsqbwsha2ap65dh09f9v0q0v43d91-python-py-1.10.0/lib/python3.9/site-packages:/gnu/store/jl1g2qqlg9sjxrh649x8zp2ysy4fmwh5-python-pluggy-0.13.1/lib/python3.9/site-packages:/gnu/store/driz9p0pv29s9dlpd7r8m5r65wiia30z-python-more-itertools-8.2.0/lib/python3.9/site-packages:/gnu/store/sww1f0qbddpnj7p1pivrsva83xn7c711-python-iniconfig-1.1.1/lib/python3.9/site-packages:/gnu/store/3qxwkb6ap8fqxq6jpd5biq1spvrvfx5b-python-coverage-5.2.1/lib/python3.9/site-packages:/gnu/store/y5pp1xag89515bamhc4115qv5ylfz7yc-python-werkzeug-2.0.2/lib/python3.9/site-packages:/gnu/store/5lxfyv08c9zxcvdgb44w1q5gpr5xg5xb-python-vobject-0.9.6.1/lib/python3.9/site-packages:/gnu/store/q0j9pj5f01rpabycihyaznwgp5q83vvk-python-passlib-1.7.4/lib/python3.9/site-packages:/gnu/store/qw8v1v3v3wz3rb50mi8ff3a3idiyx8wz-python-defusedxml-0.6.0/lib/python3.9/site-packages:/gnu/store/dbna3m3if9hhwgwxa8gj2x94g7i72cv2-python-dateutil-2.8.2/lib/python3.9/site-packages:/gnu/store/4c1b0lx1gf5q4ddbc7w9pfkfi247qa9d-python-yarl-1.6.3/lib/python3.9/site-packages:/gnu/store/vx0xdj70yl57n4all9li3f0kpqjpnvzv-python-typing-extensions-4.0.1/lib/python3.9/site-packages:/gnu/store/6ahckls8paydyxpz5pfy1lfxm4vc5i17-python-multidict-5.2.0/lib/python3.9/site-packages:/gnu/store/47x8agyg6l0krm3c06g2pkrfq0d0cyqx-python-idna-ssl-1.0.1/lib/python3.9/site-packages:/gnu/store/11xr17i7m4g9jzxfkj3bxhnfhal9ffk2-python-frozenlist-1.2.0/lib/python3.9/site-packages:/gnu/store/nfl65lgcc2d2xwcpnvyk8vsm964cm2wb-python-charset-normalizer-2.1.0/lib/python3.9/site-packages:/gnu/store/v04bfbsrgi8fc79z1ndnpcxmakgfl1fy-python-async-timeout-4.0.2/lib/python3.9/site-packages:/gnu/store/ck909qdyp9fvbn70fp40498jvkb11sl2-python-attrs-21.2.0/lib/python3.9/site-packages:/gnu/store/12l30nnp25rmw8ilrlgn70wjbraklamn-python-aiosignal-1.2.0/lib/python3.9/site-packages:/gnu/store/hnrpr6x349rha5v9n6b5famb8q05nkdd-python-aiodns-1.1.1/lib/python3.9/site-packages:/gnu/store/nhgc9mf33lxzxrqc6ng8gzwd102ml962-python-urllib3-1.26.9/lib/python3.9/site-packages:/gnu/store/hj74j5jjqr55qy8ldvs23rlxanr3f1l7-python-idna-3.3/lib/python3.9/site-packages:/gnu/store/92z66bri07mjy3hxg27r4s8g3jq4zy0p-python-certifi-2022.6.15/lib/python3.9/site-packages:/gnu/store/kw41zfr5s3ay2xva287lcpgzqa7bhvh1-python-pyparsing-3.0.6/lib/python3.9/site-packages:/gnu/store/3bjjwwwbniv92j0cg8kp1h5k2q3c42n3-python-six-1.16.0/lib/python3.9/site-packages:/gnu/store/j3j9nb50p6ch25xsqljanz8byvzphbvv-python-zipp-1.0.0/lib/python3.9/site-packages:/gnu/store/jm0lggk772a0kwgbqd8yp0ni67iijpv5-python-markupsafe-2.0.1/lib/python3.9/site-packages:/gnu/store/wkrmii2s5a5l1wliz9cwj7qgkgiqm6jn-python-pytz-2022.1/lib/python3.9/site-packages:/gnu/store/gxs46ikg02xhhriy2995c5h5a4vl3779-python-filelock-3.0.12/lib/python3.9/site-packages:/gnu/store/i2v7zagys88z96nyaqmz4897xpx8pwpb-python-distlib-0.3.1/lib/python3.9/site-packages:/gnu/store/5s8xr582hif70nghlfdadp8y0isly5fb-python-appdirs-1.4.3/lib/python3.9/site-packages:/gnu/store/xr7l6fkj6fwvxyvamxllsp7x0zm4j3ag-python-iso8601-1.0.2/lib/python3.9/site-packages:/gnu/store/smn8153mc4qaqrk6pw9k99gfz1gi69xm-python-cffi-1.15.1/lib/python3.9/site-packages:/gnu/store/cnzvnhjqwgiqdlyyrkrpb4vkydxm4din-python-asn1crypto-1.4.0/lib/python3.9/site-packages:/gnu/store/0bda1r8cjbjzdzjj7laq0ks04h78azli-python-pyicu-2.9/lib/python3.9/site-packages:/gnu/store/4kddkcpshdl7ypz989lzd3v1hlrrxmj5-python-bcrypt-3.2.0/lib/python3.9/site-packages:/gnu/store/x0m150gpbbsa2yvjx5gbkx237yhb5f0g-python-pycares-2.3.0/lib/python3.9/site-packages:/gnu/store/hjrz06f1hfg6m0adil7wd78qc7a58lsg-python-pysocks-1.7.1/lib/python3.9/site-packages:/gnu/store/jkc9lc7n61s0rw4mnpmflaiyb5vgnal3-python-pyopenssl-21.0.0/lib/python3.9/site-packages:/gnu/store/1mdg7xc4zx0i9s0kd0hwwq98bgab51s1-python-pycparser-2.21/lib/python3.9/site-packages" /gnu/store/slhyk2ng2nlxhz4bzqxavgsksbsgp9j6-python-wrapper-3.9.9/bin/python
Python 3.9.9 (main, Jan 1 1970, 00:00:01)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get(url="https://calendar.google.com/calendar/ical/[redacted]%40group.calendar.google.com/public/basic.ics")
<Response [200]>
>>>
We use python-aiohttp under the hood, not requests. Can you try making a request with it via an interactive shell?
OK, I checked requests per this documentation, which may need to be updated:
https://vdirsyncer.readthedocs.io/en/stable/ssl-tutorial.html
aiohttp using aiohttp.ClientSession get() and asyncio.get_event_loop().run_until_complete() fails with the same error as vdirsyncer. I will look into this further.
Thanks for pointing that out. The docs are now updated: 7c72caef3f8b1754c2c2e906bd3b4fea23091b8e. (note: this won't get published until the next release).
Sorry for the delay. I have prodded at this a few more times, and I am pretty convinced that it is a packaging or dependency error. I have filed this ticket against Guix: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=61557
I will update here as well if anything comes to light.