Snapshot Deletion Fails Due to Stuck “Backup Shipping” Queue

[boedy]

How I got here: I’m experimenting with Velero and Linstor’s snapshotting and snapshot shipping mechanism, following the guide in this article: 👉 Using Velero & LINBIT SDS to Backup & Restore Kubernetes

However, I’ve encountered an issue where a snapshot cannot be deleted. The snapshot remains in a queued state for backup shipping.

Trying to delete the snapshot manually

linstor snapshot delete pvc-782b6a0c-d3bf-4895-a982-b0bdb4379e36 snapshot-1dfff8a5-4c62-466b-90d7-dd8a76d88c8b
ERROR:
Description:
    Snapshot definition snapshot-1dfff8a5-4c62-466b-90d7-dd8a76d88c8b of resource pvc-782b6a0c-d3bf-4895-a982-b0bdb4379e36 is currently being queued for backup shipping. Please wait until the shipping is finished or use backup abort --create
Details:
    Resource: pvc-782b6a0c-d3bf-4895-a982-b0bdb4379e36, Snapshot: snapshot-1dfff8a5-4c62-466b-90d7-dd8a76d88c8b
Show reports:
    linstor error-reports show 67C5D826-00000-000044
command terminated with exit code 10

Trying to list backup queue:

velero linstor backup q l
ERROR:
Description:
    Exception thrown.
Details:
    Access to deleted SnapshotDefinition
Show reports:
    linstor error-reports show 67C5D826-00000-000047
command terminated with exit code 10

CSI-Snapshotter reports: csi-snapshotter.log

[boedy]

As listed in the error, with this command I was able to resolve the issue

linstor backup abort --create pvc-782b6a0c-d3bf-4895-a982-b0bdb4379e36-h

[ghernadi]

Many backup queue related issues, as well as most "Access to deleted " errors can in general be resolved by restarting the controller. But I am happy that you found a different way to resolve your issue.

In order to prevent such behavior in the future, can you please send an SOS report to my email (see profile)? I would like to see if I can find the root cause of the mentioned issues. Thanks.