HyperBackup fails to verify integrity

[zandadoum]

Hello, I am sure that you know what Hyper backup is, as I think you tested this on synology + docker, right?

so, I setup your proxy, I got it to work to upload the backups (100GB) every day to a sharepoint site

but once the integrity check hits on sunday, it throws an error about corrupted backups and stops working

is this something because of the proxy, because of hyperbackup or just straight away sharepoint online?

could you help with this?

[skleeschulte]

Hello, sorry for the late response. Is this still an issue? If so, first step I suggest is to turn on debug-logging and have a look at the log output.

[mg2c]

Same here: using the current docker image. Problem is: since I did the verification and it failed, I can‘t even start the synchronization anymore. I‘d love to try to overwrite/correct the possibly corrupt upload, but Hyper Backup cannot start that job anymore, even manually: all relevant “buttons” are gone. What do you suggest, what can I provide to you?

[Blubern]

Hi,

@skleeschulte thank you for the project and the nice approach. I'm facing the same issue. Like descripted here and in #11.

I see in the synology logs a after a time the error => Server error: status='429', msg = 'Too many request'

So I think it is a issue with the throttling in the OneDrive Business case. https://docs.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online

And it makes sense that in case of a Backup the issue is not so often because the number of Request to the OneDrive are not that much. And in Case of a Integrity Check where are a lot of request to OneDrive we getting the throttling effect.

I will try to add to your solution the possibility to add the UserAgent from an environment variable. And then people can add an App Registration and add the string in the correct form to every request to the OneDrive API what should then prevent us from getting throttled by the OneDrive Service.

Hopefully this will work and the issue is gone.

My Log File Entries: 2022-01-25T18:30:13+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:30:20+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:30:20+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:30:26+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:30:26+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:30:32+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:30:32+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:30:38+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:30:38+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #1: stat failed: -1000, Too many request 2022-01-25T18:30:45+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:30:45+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #2: stat failed: -1000, Too many request 2022-01-25T18:30:54+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:30:54+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #3: stat failed: -1000, Too many request 2022-01-25T18:31:15+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:31:15+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:31:21+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:31:21+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #1: stat failed: -1000, Too many request 2022-01-25T18:31:28+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:31:28+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #2: stat failed: -1000, Too many request 2022-01-25T18:31:53+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:31:53+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:31:59+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:31:59+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #1: recvFile failed: -1000, Too many request 2022-01-25T18:32:21+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:32:21+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:32:27+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:32:27+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #1: recvFile failed: -1000, Too many request 2022-01-25T18:32:50+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:32:50+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:32:56+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:32:56+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #1: recvFile failed: -1000, Too many request 2022-01-25T18:33:19+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:33:19+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:33:25+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:33:25+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #1: stat failed: -1000, Too many request 2022-01-25T18:33:47+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:33:47+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:33:53+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:33:53+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #1: recvFile failed: -1000, Too many request 2022-01-25T18:34:16+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:34:16+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:34:22+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:34:22+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #1: stat failed: -1000, Too many request 2022-01-25T18:34:29+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:34:29+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #2: stat failed: -1000, Too many request 2022-01-25T18:34:56+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:34:56+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:35:19+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:35:19+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:35:25+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:35:25+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #1: recvFile failed: -1000, Too many request 2022-01-25T18:40:29+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:40:29+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:40:52+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:40:52+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:40:59+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:40:59+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #1: recvFile failed: -1000, Too many request 2022-01-25T18:43:59+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:43:59+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:51:10+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:51:10+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:51:16+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:51:16+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:51:22+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:51:22+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:51:28+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:51:28+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:51:34+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:51:34+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:53:09+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:53:09+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:53:15+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:53:15+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:53:21+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:53:21+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:53:28+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:53:28+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:53:34+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:53:34+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:58:58+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:58:58+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:59:04+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:59:04+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:59:10+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:59:10+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:59:17+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:59:17+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request 2022-01-25T18:59:23+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(363): Server error: status='429', msg = 'Too many request' 2022-01-25T18:59:23+01:00 synostation img_backup[14512]: transfer_webdav.cpp:417 need retry #0: stat failed: -1000, Too many request 2022-01-25T18:59:29+01:00 synostation img_backup[14512]: cloudstorage/protocol/webdav/webdav-protocol.cpp(139): Server error: status='429', msg = 'Too many request' 2022-01-25T18:59:29+01:00 synostation img_backup[14512]: transfer_webdav.cpp:326 need retry #0: recvFile failed: -1000, Too many request

[cuongpl]

I'm facing the same issue. @Blubern

I will try to add to your solution the possibility to add the UserAgent from an environment variable. And then people can add an App Registration and add the string in the correct form to every request to the OneDrive API what should then prevent us from getting throttled by the OneDrive Service.

Is there any update on it?

[Blubern]

@cuongpl I was trying a lot to make it work, without success!

[cuongpl]

@Blubern Thanks for your reply. I temporarily turned off HyperBackup integrity check.