Upload to S3 not finished during one backup time window creates large amount of traffic

[Hoffmann77]

Hey there. I am experimenting with the backup app since a couple of weeks now. I want to use it in combination with a S3 external storage. In the last weeks I have experienced quite the amount of data traffic generated by the backup app, which corresponds to #162. It's not perfect for a private internet connection, but it's understandable why it's done this way.

Another behavior I have experienced is occurring when the upload of a backup point isn't finished during the maintenance time window. It seems like all already uploaded backup data is downloaded during the next maintenance time window just before uploading to S3 continues. As the backup on the external storage gets larger during every maintenance time window, the data getting downloaded during the next maintenance time window gets larger as well. As a consequence the time left to upload new data gets shorter every time and a large amount of traffic is generated. Is this an expected behavior or a misconfiguration/bug issue?

In the image below you can see ~40gb of backup-data getting downloaded just before uploading continues:

[AnAnalogGuy]

A bit OT: If you want to reduce network traffic, you might want to think of doing a local backup and then use restic to transfer to S3. Like this you get the benefit of deduplication which will recude traffic and storage usage. Restic also works with S3 and provides encryption.

https://restic.net/