problem downloading from UDM PRO, 2 cameras

[Johnny5AtGithub]

Hi, I may of course have made a mistake in the startup command, but since it does read the camera list I think a connection is indeed possible, but later on when downloading video it fails due to a timeout. Running this on a Raspberry Pi 4b 4gb ram. OS is Ubuntu 20.04 LTS (GNU/Linux 5.4.0-1041-raspi aarch64)

`ubuntu@NASpberryMedia1:~$ docker run --name unifitoolbox --rm --volume /Data/unifitoolbox:/downloads unifitoolbox/protect-archiver download --address 192.168.0.1 --username USERNAME --password PASSWORD --cameras all --wait-between-downloads 1 --ignore-failed-downloads --skip-existing-files --use-subfolders --start 2021-08-01 --end 2021-08-02 /downloads Getting camera list Successfully authenticated as user USERNAME using a session cookie Successfully retrieved data from https://192.168.0.1:443/proxy/protect/api/cameras Cameras found:

• Doorbell (60ef36da03a43a03870006b7)
• G3 Instant (6107c8fc00877b0387001d40)

Downloading video files between 2021-08-01 00:00:00 and 2021-08-02 00:00:00 from 'https://192.168.0.1:443/proxy/protect/api/video/export' for camera Doorbell Downloading footage for camera 'Doorbell' (60ef36da03a43a03870006b7) Downloading video for time range 2021-08-01 00:00:00 - 2021-08-01 00:59:59 to /downloads/2021/08/01/Doorbell (06b7)/Doorbell (06b7) - 2021-08-01 - 00.00.00.mp4 Download failed: HTTPSConnectionPool(host='192.168.0.1', port=443): Read timed out. (read timeout=60.0) Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/usr/local/lib/python3.8/http/client.py", line 1344, in getresponse response.begin() File "/usr/local/lib/python3.8/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/local/lib/python3.8/http/client.py", line 268, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/local/lib/python3.8/socket.py", line 669, in readinto return self._sock.recv_into(b) File "/usr/local/lib/python3.8/ssl.py", line 1241, in recv_into return self.read(nbytes, buffer) File "/usr/local/lib/python3.8/ssl.py", line 1099, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 755, in urlopen retries = retries.increment( File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 532, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise raise value File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 447, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 336, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='192.168.0.1', port=443): Read timed out. (read timeout=60.0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/protect_archiver/downloader/download_file.py", line 31, in download_file response = requests.get( File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, **kwargs) File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 529, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='192.168.0.1', port=443): Read timed out. (read timeout=60.0) Retrying in 3 second(s)... Download failed: HTTPSConnectionPool(host='192.168.0.1', port=443): Read timed out. (read timeout=60.0) Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/usr/local/lib/python3.8/http/client.py", line 1344, in getresponse response.begin() File "/usr/local/lib/python3.8/http/client.py", line 307, in begin version, status, reason = self._read_status() File "/usr/local/lib/python3.8/http/client.py", line 268, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/usr/local/lib/python3.8/socket.py", line 669, in readinto return self._sock.recv_into(b) File "/usr/local/lib/python3.8/ssl.py", line 1241, in recv_into return self.read(nbytes, buffer) File "/usr/local/lib/python3.8/ssl.py", line 1099, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out`

[danielfernau]

Hi @Johnny5AtGithub – apologies for the delayed response.

I'm not entirely sure what causes this... As you've already mentioned, based on the logs the tool is able to log in and read the camera list, meaning that the communication on network level as well as API access and UDM local user permissions seem to be working fine. It might be that either the Raspberry Pi or the UDM are a bit slow for some reason... You can try increasing the download request timeout by adding the --download-request-timeout option to your command line, e.g. --download-request-timeout 120, and see if that makes a difference.

If that doesn't work let me know and we can try a few other things.

[Johnny5AtGithub]

I tried your addition. I get the errors below very fast. Then I changed "--wait-between-downloads 1" to 120 seconds and then the output below came, exact same log, just a lot slower :D When I download via the web interface in a time slot of 1 hour, it at least started well within the 2 minutes timeout. Since http500 suggests something is going wrong in the UDM, but all other operations including manually downloading work fine. I'm adding the manual download URL that works in my browser, maybe the latest UDM software version changed something that broke your tool? "https://dreammachine.local/proxy/protect/api/video/export?channel=0&filename=G3%20Instant%20-%202021-08-07_19.00.mp4&camera=6107c8fc00877b0387001d40&start=1628362800000&end=1628366400000"

ubuntu@NASpberryMedia1:~$ docker run --name unifitoolbox --rm --volume /Data/unifitoolbox:/downloads unifitoolbox/protect-archiver download --address dreammachine.local --username USERNAME --password PASSWORD --cameras all --wait-between-downloads 120 --use-subfolders --start 2021-08-01 --end 2021-08-02 --download-request-timeout 120 /downloads Getting camera list Successfully authenticated as user USERNAME using a session cookie Successfully retrieved data from https://dreammachine.local:443/proxy/protect/api/cameras Cameras found:

• G3 Instant (6107c8fc00877b0387001d40)
• Doorbell (60ef36da03a43a03870006b7)

Downloading video files between 2021-08-01 00:00:00 and 2021-08-02 00:00:00 from 'https://dreammachine.local:443/proxy/protect/api/video/export' for camera G3 Instant Downloading footage for camera 'G3 Instant' (6107c8fc00877b0387001d40) Downloading video for time range 2021-08-01 00:00:00 - 2021-08-01 00:59:59 to /downloads/2021/08/01/G3 Instant (1d40)/G3 Instant (1d40) - 2021-08-01 - 00.00.00.mp4 Download failed with status 500 Internal Server Error: No files found matching criteria. NoneType: None Downloading video for time range 2021-08-01 01:00:00 - 2021-08-01 01:59:59 to /downloads/2021/08/01/G3 Instant (1d40)/G3 Instant (1d40) - 2021-08-01 - 01.00.00.mp4 Download failed with status 500 Internal Server Error: No files found matching criteria. NoneType: None Downloading video for time range 2021-08-01 02:00:00 - 2021-08-01 02:59:59 to /downloads/2021/08/01/G3 Instant (1d40)/G3 Instant (1d40) - 2021-08-01 - 02.00.00.mp4 Download failed with status 500 Internal Server Error: No files found matching criteria. NoneType: None Downloading video for time range 2021-08-01 03:00:00 - 2021-08-01 03:59:59 to /downloads/2021/08/01/G3 Instant (1d40)/G3 Instant (1d40) - 2021-08-01 - 03.00.00.mp4 Download failed with status 500 Internal Server Error: No files found matching criteria. NoneType: None

[danielfernau]

Yeah, well... --wait-between-downloads pretty much does exactly what it's name implies... 😄 But under normal circumstances you don't need that one, so you could theoretically remove it.

Now that we know that it's not actually a timeout issue I would think that the channel=0 in your manual download URL might somehow be important for the UDM to be able to start the export correctly. I haven't tested/used the tool with G3 Instant cameras yet... Maybe they use multiple recording channels and the missing parameter is what breaks the download?

Let's try something: could you change your command slightly to download clips for just your Doorbell using --cameras="60ef36da03a43a03870006b7"? If that works as expected I might need to update the tool to add support for multiple recording channels...

[Johnny5AtGithub]

Hmmm that is interesting. Initially I thought the doorbell was the problem since the movie it first downloads is in the night and when manually downloading an hour in the night resulted in a 2gb file. But when I ran the command for the 1 camera, it ran fine! So naturally i rand the same command for the other camera and that runs fine as well!!

So seems there is something wrong running for both camera's in 1 command. Well, at least there is a workaround available now. Running 2 commands is still acceptable. Let me know if you want help debugging

Addendum, if I select both cameras via comma separated ID's it also works just fine.

[danielfernau]

Interesting for sure! Thanks again for sharing your observations – I'll see what I can do once there's a bit more free time on my end. Sounds a bit like the API is returning additional/unusable/invalid camera IDs if they're not manually provided.

I'll leave this issue open for now and add updates if there's any progress to share. Also, good to hear that you've found a workaround!

[knarfknarf]

@danielfernau Thanks for this great tool!

I am seeing similar issue as Johnny above has faced. Differences are I am using the sync function, not using docker, and running Protect on UNVR. It's ran on a M1 Mac mini with homebrew python 3.9. UNVR is on latest release version 1.19.0.

Below are the outputs. This would happen pretty much on every download (I have 12 cameras, set to sync footage every 6 hours). While this fails, the UNVR would become unavailable so all my cameras would not be able to record during the error happens. It can range from a few minutes to over 10 minutes. I tried both the --cameras=all and --cameras="ID1,ID2,etc" methods and got the same error.

This is my sync command: ./protect-archiver sync --address 10.0.100.2 --username USERNAME --password PASSWORD --cameras=all --ignore-failed-downloads /Volumes/myDataset/Protect

Download failed: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0) Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1349, in getresponse response.begin() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 316, in begin version, status, reason = self._read_status() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 277, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto return self._sock.recv_into(b) File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1241, in recv_into return self.read(nbytes, buffer) File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1099, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 755, in urlopen retries = retries.increment( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/util/retry.py", line 532, in increment raise six.reraise(type(error), error, _stacktrace) File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/packages/six.py", line 770, in reraise raise value File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 447, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 336, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/downloader/download_file.py", line 31, in download_file response = requests.get( File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/api.py", line 75, in get return request('get', url, params=params, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/adapters.py", line 529, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0) Retrying in 3 second(s)... Download failed: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0) Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1349, in getresponse response.begin() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 316, in begin version, status, reason = self._read_status() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 277, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto return self._sock.recv_into(b) File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1241, in recv_into return self.read(nbytes, buffer) File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1099, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 755, in urlopen retries = retries.increment( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/util/retry.py", line 532, in increment raise six.reraise(type(error), error, _stacktrace) File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/packages/six.py", line 770, in reraise raise value File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 447, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 336, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/downloader/download_file.py", line 31, in download_file response = requests.get( File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/api.py", line 75, in get return request('get', url, params=params, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/adapters.py", line 529, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0) Retrying in 3 second(s)... Download failed: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0) Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1349, in getresponse response.begin() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 316, in begin version, status, reason = self._read_status() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 277, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto return self._sock.recv_into(b) File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1241, in recv_into return self.read(nbytes, buffer) File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1099, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 755, in urlopen retries = retries.increment( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/util/retry.py", line 532, in increment raise six.reraise(type(error), error, _stacktrace) File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/packages/six.py", line 770, in reraise raise value File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 447, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 336, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/downloader/download_file.py", line 31, in download_file response = requests.get( File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/api.py", line 75, in get return request('get', url, params=params, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/adapters.py", line 529, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0) Retrying in 3 second(s)... Argument '--ignore-failed-downloads' is present, continue downloading files... Downloading footage for camera 'Front Yard Pathway' (60cfaa07021f5d03e7001717) Downloading video for time range 2021-08-17 12:00:00 - 2021-08-17 12:59:58 to /Volumes/myDataset/Protect/2021/08/17/Front Yard Pathway (1717)/Front Yard Pathway (1717) - 2021-08-17 - 12.00.00.mp4 Download failed: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0) Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 445, in _make_request six.raise_from(e, None) File "", line 3, in raise_from File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 440, in _make_request httplib_response = conn.getresponse() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1349, in getresponse response.begin() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 316, in begin version, status, reason = self._read_status() File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 277, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto return self._sock.recv_into(b) File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1241, in recv_into return self.read(nbytes, buffer) File "/opt/homebrew/Cellar/[email protected]/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1099, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/adapters.py", line 439, in send resp = conn.urlopen( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 755, in urlopen retries = retries.increment( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/util/retry.py", line 532, in increment raise six.reraise(type(error), error, _stacktrace) File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/packages/six.py", line 770, in reraise raise value File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 447, in _make_request self._raise_timeout(err=e, url=url, timeout_value=read_timeout) File "/opt/homebrew/lib/python3.9/site-packages/urllib3-1.26.6-py3.9.egg/urllib3/connectionpool.py", line 336, in _raise_timeout raise ReadTimeoutError( urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/downloader/download_file.py", line 31, in download_file response = requests.get( File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/api.py", line 75, in get return request('get', url, params=params, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/api.py", line 61, in request return session.request(method=method, url=url, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/sessions.py", line 542, in request resp = self.send(prep, **send_kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/sessions.py", line 655, in send r = adapter.send(request, **kwargs) File "/opt/homebrew/lib/python3.9/site-packages/requests-2.26.0-py3.9.egg/requests/adapters.py", line 529, in send raise ReadTimeout(e, request=request) requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='10.0.100.2', port=443): Read timed out. (read timeout=60.0) Retrying in 3 second(s)... Failed to sync camera Front Yard Pathway - continuing to next device Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/sync.py", line 56, in run Downloader.download_footage(self.client, interval_start, interval_end, camera) File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/downloader/init.py", line 41, in download_footage return download_footage(client, start, end, camera) File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/downloader/download_footage.py", line 72, in download_footage download_file(client, video_export_query, filename) File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/downloader/download_file.py", line 61, in download_file data.get("error") or data or "(no information available)" AttributeError: 'NoneType' object has no attribute 'get' Downloading footage for camera 'Front Yard' (600df1fc03a70903e700099b) Downloading video for time range 2021-08-17 05:00:00 - 2021-08-17 05:59:58 to /Volumes/myDataset/Protect/2021/08/17/Front Yard (099b)/Front Yard (099b) - 2021-08-17 - 05.00.00.mp4 Failed to sync camera Front Yard - continuing to next device Traceback (most recent call last): File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/sync.py", line 56, in run Downloader.download_footage(self.client, interval_start, interval_end, camera) File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/downloader/init.py", line 41, in download_footage return download_footage(client, start, end, camera) File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/downloader/download_footage.py", line 72, in download_footage download_file(client, video_export_query, filename) File "/opt/homebrew/lib/python3.9/site-packages/protect_archiver-2.0.2-py3.9.egg/protect_archiver/downloader/download_file.py", line 61, in download_file data.get("error") or data or "(no information available)" AttributeError: 'NoneType' object has no attribute 'get' 112 files downloaded (123.1 gb), 1 files skipped, 0 files failed, 113 files total

[Johnny5AtGithub]

I don't remember exactly if it is available for the sync feature but i could try and set the timeout to 120 seconds. I needed to do that because 1 hour of night footage from the doorbell was 2gb and the protect controller takes too long to respond with the data

[AshDevFr]

Hi, I have been using the event feature without any issue for some time but I experience the same issue as other people with download and sync

I am using the docker image.

[danielfernau]

Hi @AshDevFr, thanks for the feedback. The clips with which the event feature interacts are a lot shorter than the regular continuous recordings which can be up to around 1h at the moment. This leads me to believe that the main problem with the timeouts might actually be on the UDM/API side. Maybe it's too slow to finish converting the content before the downloader thinks that there's no response... Or maybe it gets stuck and then runs into the timeout... Not sure. I haven't experienced this issue myself yet, which makes it a bit more complicated to solve. I'll see what I can do based on the logs and details provided by everyone in this thread.

[adams-family]

I had the same timeout issue and wanted to confirm that --download-request-timeout 120 did help me solve the issue. Although, my CKG2+ is so slow that I think I need even more, as downloading an hour worth of video takes about ±350s.