csi-driver-lvm icon indicating copy to clipboard operation
csi-driver-lvm copied to clipboard

Perform LV activation/deactivation

Open kvaps opened this issue 1 year ago • 3 comments

This PR introduces:

  • Deactivation of volume after lvcreate
  • Activation of volume before usage
  • Deactivation of volume after usage
  • sync+lvscan before every lvm operation

This is first step needed to implement shared vg support https://github.com/metal-stack/csi-driver-lvm/issues/62

kvaps avatar Apr 28 '23 14:04 kvaps

Hi @kvaps

I still do not understand how lvm-ha setup would look like ? Can you explain a bit more who would be responsible to sync data between nodes and how.

majst01 avatar May 02 '23 08:05 majst01

Hey, sorry, for insufficient description to this PR. I'm solving the issue of using LVM over shared storage (DAS).

Eg. when you have shared LUN connected to multiple nodes in your cluster. image

The LVM is used to cut this volume on small pieces I like design of your driver, and I'm going to develop my own own based on it.

I wish to contribute changes back, but not sure if you'll agree with these changes.

  • First of all I want driver which will work in both cases: Local LVM and Shared LVM.
  • The driver should have configurable volume group in StorageClass, thus it can work with many VGs.
  • Volumes should be created with nodeAffinity to VG's, not to the nodes.
  • For each VG driver should add label to node with UUID of this VG. Thus if VG is shared across the multuple nodes the Volume can be used on any of them.
  • The driver should not use cLVM or lvmlockd extensions, they are pretty complex and difficult to maintain. Instead it must to refresh LVM metadata before the each operation on VG. The locks should be implemented using Kubernetes API.
  • Snapshots support

The design of shared LVM without cLVM and lvmlockd is borrowed from such solutions as OpenNebula and Proxmox. Both do not require the clustered extension for LVM:

proxmox:

You don't need CLVM as long as you use the pve tools/API to manage volumes (create/delete).

opennebula:

The LVM datastore does not need CLVM configured in your cluster. The drivers refresh LVM metadata each time an image is needed on another Hosts.

here are some excerpts from the LVM mailing list:

cLVM does not control concurrent access, it just cares about propagating the lvm metadata to all nodes and locking during changes of the metadata.

All clustered LVM does is make sure that, as LVM things change, all nodes know about the changes immediately.

  • https://lists.clusterlabs.org/pipermail/users/2015-December/018263.html
  • https://www.spinics.net/lists/lvm/msg21913.html
  • https://ivirt-it.ru/clvm-lvmlockd/

Unfortunately, I have no enough time to continue with this right now. It's better to convert it to draft. Feel free to leave any comments on my design.

kvaps avatar May 03 '23 08:05 kvaps

This seems a lot of effort and does not fit well in our use case. It is probably better to take this code and create your own csi driver. But if you have small improvements for this one, they are all welcome.

majst01 avatar May 03 '23 08:05 majst01