unifi-protect-backup icon indicating copy to clipboard operation
unifi-protect-backup copied to clipboard

rclone is SLOW on WebDAV

Open cyberpower678 opened this issue 2 months ago • 3 comments

Despite setting the transfers option to 32 parallel transfers, and the logs are confirming that rclone has set the --transfers option to 32, and also setting the NextCloud chunk size to 1 Gi, also confirmed by the logs, uploading to NextCloud is so painfully slow, the buffer backs up almost immediately, despite NextCloud living in a neighboring k3s pod, ie on the same machine. Latency between them is 3 microseconds, and obviously network stack IO is limited by how fast the CPU can move the data through the k3s cluster.

Setting the option appears to do nothing as it seems to me the uploads are still happening one at a time. Can something be done about this. Transfer speeds are only 25 Mbps to NextCloud which is quite awful for this setup. Camera events are often many little files rather than few big files making the opening and closing of the connection/sockets more limiting than the bandwidth. As such parallel uploads would be immensely useful.

cyberpower678 avatar Jun 09 '24 14:06 cyberpower678