Derek Su

@mantissahz Need to update v1.8.0-dev and v1.6.3-dev as well. Thank you.

nit: The volume or snapshot size is too small to observe any changes in size during the backup process.

@roger-ryao Can you help revalidate? Thank you.

@PortableProgrammer Can you provide a support bundle when you run into the error? The issue might be caused by lock acquisition issue when network is partitioned or other issues. Need...

BTW, I saw some log messages in the log ``` 2023-06-22T00:31:04.251538081-06:00 time="2023-06-22T06:31:04Z" level=error msg="Failed to run job for volume" concurrent=1 error="failed to complete backupAndCleanup for pvc-7a165daf-3376-401f-a38c-73ca8a806c1d: failed to create backup...

@PortableProgrammer Thanks for your support bundle. I saw the log messages in your support replica-manager.log. ``` 2023-06-22T00:35:42.774444959-06:00 [pvc-79b70ecd-d85d-486f-aa30-2a33dc384598-r-6becd5ef] time="2023-06-22T06:35:42Z" level=info msg="Initializing backup backup-754a313362414f44 for volume pvc-79b70ecd-d85d-486f-aa30-2a33dc384598 snapshot daily-ba-419ce990-6417-46d0-934d-6ee1a0e2186a" pkg=backup 2023-06-22T00:35:42.774494059-06:00...

> I'm reviewing the NFS logs to see what was going on during the few seconds between the initial `permission denied` lock rename and the success. The fact that it...

@PortableProgrammer Thanks for the update. Sorry for the late reply. Please let me know if it happens again and I will continue investigating the issue from your support bundles.

@Androz2091 It might be related to https://github.com/longhorn/longhorn/issues/9168. @ChanYiLin Can you help confirm it? Thanks.

@Androz2091 For sure, feel free to open a new ticket with the information for investigation. Thank you.