Does not properly follow redirects on CalDAV URLs

[madduck]

trafficstars

Hey, I'm using 0.16.7 (Debian sid) and running against DAViCal, although it's behind a redirecting URL setup. Basically, https://edsfellas.nz/caldav forwards to https://calendar.edsfellas.nz/caldav. The /.well-known URLs also work similarly.

I could use the target URL directly, sure, but the idea here is to be able to use a stable URL, if the actual server moves, the original URL should still work. That's the idea anyway.

The problem with this setup is that the target server returns a 401, so vdirsyncer responds to the HTTP Basic-Auth with the password… but it submits those to the original server, which redirects, and that yields the same 401 again. This is not the behaviour that the 307 redirect HTTP status code demands, but instead, vdirsyncer should submit the credentials to the target URL only.

Here is the config file:

[pair ehf]
a = "ehf_local"
b = "ehf_remote"
collections = ["from b"]

[storage ehf_local]
type = "filesystem"
path = "~/.var/vdirsyncer/ehf"
fileext = ".ics"

[storage ehf_remote]
type = "caldav"
url = "https://edsfellas.nz/caldav"
username = "[email protected]"
password.fetch = ["command","~/.config/vdirsyncer/getpw.sh","ehf_remote"]
verify = "/etc/ssl/certs/ca-certificates.crt"

And here are the logs:

lotus:~% vdirsyncer -vdebug discover ehf                                                                                                                            #(1 pts/4) 30 13:21:31.819
debug: Using 1 maximal workers.
debug: Fetching value for password.fetch with command strategy.
Discovering collections for pair ehf
ehf_local:
  - "events"
debug: PROPFIND https://edsfellas.nz/caldav/
debug: {'User-Agent': 'vdirsyncer/0.16.7', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: 
debug:     <d:propfind xmlns:d="DAV:">
debug:         <d:prop>
debug:             <d:resourcetype />
debug:         </d:prop>
debug:     </d:propfind>
debug:     
debug: Sending request...
debug: Rewriting status code from 307 to 307
debug: 401
debug: {'Date': 'Thu, 30 Apr 2020 01:21:34 GMT', 'Server': 'Apache/2.4.38 (Debian)', 'WWW-Authenticate': 'Basic realm="EHF Community CalDAV Server"', 'Content-Length': '40', 'Keep-Alive': 'timeout=5, max=100', 'Connection': 'Keep-Alive', 'Content-Type': 'text/plain; ; charset="utf-8"'}
debug: Please log in for access to this system.
debug: Given URL is not a homeset URL
debug: PROPFIND https://edsfellas.nz/caldav/
debug: {'User-Agent': 'vdirsyncer/0.16.7', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '0'}
debug: 
debug:         <d:propfind xmlns:d="DAV:">
debug:             <d:prop>
debug:                 <d:current-user-principal />
debug:             </d:prop>
debug:         </d:propfind>
debug:         
debug: Sending request...
debug: Rewriting status code from 307 to 307
debug: 401
debug: {'Date': 'Thu, 30 Apr 2020 01:21:34 GMT', 'Server': 'Apache/2.4.38 (Debian)', 'WWW-Authenticate': 'Basic realm="EHF Community CalDAV Server"', 'Content-Length': '40', 'Keep-Alive': 'timeout=5, max=99', 'Connection': 'Keep-Alive', 'Content-Type': 'text/plain; ; charset="utf-8"'}
debug: Please log in for access to this system.
debug: Trying out well-known URI
debug: PROPFIND https://edsfellas.nz/.well-known/caldav
debug: {'User-Agent': 'vdirsyncer/0.16.7', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '0'}
debug: 
debug:         <d:propfind xmlns:d="DAV:">
debug:             <d:prop>
debug:                 <d:current-user-principal />
debug:             </d:prop>
debug:         </d:propfind>
debug:         
debug: Sending request...
debug: Rewriting status code from 307 to 307
debug: Rewriting status code from 301 to 307
debug: 401
debug: {'Date': 'Thu, 30 Apr 2020 01:21:35 GMT', 'Server': 'Apache/2.4.38 (Debian)', 'WWW-Authenticate': 'Basic realm="EHF Community CalDAV Server"', 'Content-Length': '40', 'Keep-Alive': 'timeout=5, max=97', 'Connection': 'Keep-Alive', 'Content-Type': 'text/plain; ; charset="utf-8"'}
debug: Please log in for access to this system.
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/discover.py", line 204, in _print_collections
debug:     discovered = get_discovered()
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/discover.py", line 133, in get_self
debug:     return self._discovered
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/utils.py", line 170, in __get__
debug:     obj.__dict__[self.__name__] = result = self.fget(obj)
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/discover.py", line 142, in _discovered
debug:     return handle_storage_init_error(self._cls, self._config)
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/discover.py", line 138, in _discovered
debug:     discovered = list(self._cls.discover(**self._config))
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 267, in discover
debug:     for c in self.find_collections():
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 227, in find_collections
debug:     return self._find_collections_impl(self.find_home())
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 203, in find_home
debug:     url = self.find_principal()
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 174, in find_principal
debug:     return self._find_principal_impl(self._well_known_uri)
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 187, in _find_principal_impl
debug:     response = self.session.request('PROPFIND', url, headers=headers,
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 390, in request
debug:     return http.request(method, url, session=self._session, **more)
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/http.py", line 163, in request
debug:     r.raise_for_status()
debug:   File "/usr/lib/python3/dist-packages/requests/models.py", line 941, in raise_for_status
debug:     raise HTTPError(http_error_msg, response=self)
warning: Failed to discover collections for ehf_remote, use `-vdebug` to see the full traceback.
debug: PROPFIND https://edsfellas.nz/caldav/
debug: {'User-Agent': 'vdirsyncer/0.16.7', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: 
debug:     <d:propfind xmlns:d="DAV:">
debug:         <d:prop>
debug:             <d:resourcetype />
debug:         </d:prop>
debug:     </d:propfind>
debug:     
debug: Sending request...
debug: Rewriting status code from 307 to 307
debug: 401
debug: {'Date': 'Thu, 30 Apr 2020 01:21:37 GMT', 'Server': 'Apache/2.4.38 (Debian)', 'WWW-Authenticate': 'Basic realm="EHF Community CalDAV Server"', 'Content-Length': '40', 'Keep-Alive': 'timeout=5, max=100', 'Connection': 'Keep-Alive', 'Content-Type': 'text/plain; ; charset="utf-8"'}
debug: Please log in for access to this system.
debug: Given URL is not a homeset URL
debug: PROPFIND https://edsfellas.nz/caldav/
debug: {'User-Agent': 'vdirsyncer/0.16.7', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '0'}
debug: 
debug:         <d:propfind xmlns:d="DAV:">
debug:             <d:prop>
debug:                 <d:current-user-principal />
debug:             </d:prop>
debug:         </d:propfind>
debug:         
debug: Sending request...
debug: Rewriting status code from 307 to 307
debug: 401
debug: {'Date': 'Thu, 30 Apr 2020 01:21:38 GMT', 'Server': 'Apache/2.4.38 (Debian)', 'WWW-Authenticate': 'Basic realm="EHF Community CalDAV Server"', 'Content-Length': '40', 'Keep-Alive': 'timeout=5, max=99', 'Connection': 'Keep-Alive', 'Content-Type': 'text/plain; ; charset="utf-8"'}
debug: Please log in for access to this system.
debug: Trying out well-known URI
debug: PROPFIND https://edsfellas.nz/.well-known/caldav
debug: {'User-Agent': 'vdirsyncer/0.16.7', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '0'}
debug: 
debug:         <d:propfind xmlns:d="DAV:">
debug:             <d:prop>
debug:                 <d:current-user-principal />
debug:             </d:prop>
debug:         </d:propfind>
debug:         
debug: Sending request...
debug: Rewriting status code from 307 to 307
debug: Rewriting status code from 301 to 307
debug: 401
debug: {'Date': 'Thu, 30 Apr 2020 01:21:39 GMT', 'Server': 'Apache/2.4.38 (Debian)', 'WWW-Authenticate': 'Basic realm="EHF Community CalDAV Server"', 'Content-Length': '40', 'Keep-Alive': 'timeout=5, max=97', 'Connection': 'Keep-Alive', 'Content-Type': 'text/plain; ; charset="utf-8"'}
debug: Please log in for access to this system.
error: Unknown error occured: 401 Client Error: Unauthorized for url: https://calendar.edsfellas.nz/davical/caldav.php/
error: Use `-vdebug` to see the full traceback.
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/utils.py", line 341, in _worker
debug:     func(wq=self)
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/tasks.py", line 82, in discover_collections
debug:     rv = collections_for_pair(pair=pair, **kwargs)
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/discover.py", line 75, in collections_for_pair
debug:     rv = list(expand_collections(
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/discover.py", line 163, in expand_collections
debug:     collections = get_b_discovered()
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/discover.py", line 133, in get_self
debug:     return self._discovered
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/utils.py", line 170, in __get__
debug:     obj.__dict__[self.__name__] = result = self.fget(obj)
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/discover.py", line 142, in _discovered
debug:     return handle_storage_init_error(self._cls, self._config)
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/cli/discover.py", line 138, in _discovered
debug:     discovered = list(self._cls.discover(**self._config))
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 267, in discover
debug:     for c in self.find_collections():
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 227, in find_collections
debug:     return self._find_collections_impl(self.find_home())
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 203, in find_home
debug:     url = self.find_principal()
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 174, in find_principal
debug:     return self._find_principal_impl(self._well_known_uri)
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 187, in _find_principal_impl
debug:     response = self.session.request('PROPFIND', url, headers=headers,
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/storage/dav.py", line 390, in request
debug:     return http.request(method, url, session=self._session, **more)
debug:   File "/usr/lib/python3/dist-packages/vdirsyncer/http.py", line 163, in request
debug:     r.raise_for_status()
debug:   File "/usr/lib/python3/dist-packages/requests/models.py", line 941, in raise_for_status
debug:     raise HTTPError(http_error_msg, response=self)
error: 1 out of 1 tasks failed.