ceph-iscsi-cli icon indicating copy to clipboard operation
ceph-iscsi-cli copied to clipboard

Published 20 hours ago • verified publisher icon ceph

create snapshot of rbd image failed with errno 30

Open zhuozh opened this issue 7 years ago • 6 comments

When a rbd image is mapped as a LUN, the tcmu-runner will hold the image's exclusive-lock. If we create a snapshot of this image, it will fail with errno 30 (EROFS: Read-only file system) Is there any plan or suggestion on how to support rbd snapshot without unbind the LUN, for a reseaon that we may running a VM on this LUN. VM be stopped or crashed is unacceptable when creating snapshot. #70 is releated to this issue.

zhuozh avatar Jan 05 '18 10:01 zhuozh

Hey,

We have sort of discussed in this internally, but any low level details.

I think we would like a gwcli command that calls into ceph-iscsi-config to coordinate the tcmu-runner access and do the ceph/rbd lib calls to do the snapshot.

If you have any ideas, post them!

mikechristie avatar Jan 05 '18 19:01 mikechristie

@zhuozh There shouldn't be any restriction for creating a snapshot when the image's exclusive lock is owned by another client. The client requesting the snapshot create will send an RPC message to the lock owner to create the snapshot on its behalf. Do you have a reproducer for this behavior? I just tried it in my environment and it worked.

dillaman avatar Mar 15 '18 23:03 dillaman

@dillaman My expression is a little misleading. Actually the read only error happened when rolling back an image to a snapshot.

  1. Create a LUN, mkfs and mount it on initiator node.
  2. Write some data into mounted device.
  3. Create a snapshot using rbd command
  4. iscsiadm logout, unbind clientlun through gwcli.
  5. Roll back to a snapshot, it fails with read-only error.

zhuozh avatar Mar 21 '18 10:03 zhuozh

@zhuozh That was originally a deliberate decision on the part of librbd since we really didn't expect a valid use-case where you should rollback with an alive client. If tcmu-runner knew when the last client disconnected, it could release the lock (but I have a feeling it doesn't know). Instead, as an alternative, we could add a --force optional to the rbd CLI to break the lock before a rollback.

dillaman avatar Mar 21 '18 11:03 dillaman

Adding @pcuzner

I think Paul wanted to add gwcli commands for snapshots that coordinated with the target/runner to handle this case for the user.

mikechristie avatar Mar 22 '18 00:03 mikechristie

See new PR https://github.com/ceph/ceph-iscsi-cli/pull/93

dillaman avatar Apr 30 '18 20:04 dillaman