Support for shared filesystems

[dchirikov]

Instead of spamming in #174 I will create another PR.

This is an attempt to bring shared file systems support.

sharedFileSystemPath allows the provisioner to use a filesystem that is mounted on all nodes at the same time. In this case all access modes are supported: ReadWriteOnce, ReadOnlyMany and ReadWriteMany for storage claims.

In addition volumeBindingMode: Immediate can be used in StorageClass definition.

Please note that nodePathMap and sharedFileSystemPath are mutually exclusive. If sharedFileSystemPath is used, then nodePathMap must be set to [].

[weisjohn]

@innobead any idea if something like this could land?

[weishiuchang]

Would love to see this included as well. Support for shared filesystem hostPath would be amazing especially for reusing existing shared file system environments that do not have a csi or ability to retool just yet.

[erulabs]

Hate to bump, but we would also love if this could land - would remove a huge amount of complexity on our side. Are there any known blockers @dchirikov?

[dchirikov]

@erulabs I can not merge in this repo. So it's all up to the owner. I can only say there is literally zero comments from rancher team, so not sure if it ever will be merged. Also I would like to have solved #182 too, because it's typical in our setups have shared FS and unprivileged users having local/ssh access to the nodes.

[gabbler97]

Can we merge dchirikov's solution? It would be very useful.

[derekbit]

@dchirikov Could you please resolve the conflict first? I will review it after resolving the conflict. Thanks.

[dchirikov]

@derekbit I will. Sorry pretty busy atm, give me couple of weeks, please.

[derekbit]

@dchirikov Sure thing. Take your time.

[dchirikov]

@derekbit Happy new year! I rebased my PR, please check.

[fastlorenzo]

Thanks! If anyone wants to try it, I've bundled the changes in a container at ghcr.io/fastlorenzo/local-path-provisioner:5888a9b-amd64

After creating a new cluster with k3d, you can run the following:

kubectl create configmap local-path-config --from-file config.json -n kube-system -o yaml --dry-run | kubectl apply -f -
kubectl -n kube-system get deployments.apps local-path-provisioner -o yaml | sed 's/image: .*local-path-provisioner.*$/image: ghcr.io\/fastlorenzo\/local-path-provisioner:5888a9b-amd64/g' | kubectl apply -f -

Where the content of config.json is:

{
    "sharedFileSystemPath": "/var/lib/rancher/k3s/storage",
    "nodePathMap": []
}

This will update the config of the local-path-provisioner deployment and switch the image to the version containing the support for shared file system \o/

So far, so good, didn't notice any issue yet (as long as you use ReadWriteMany mode, but this is expected). If I stop one of the node, it correctly switches the pod to another node without re-creating a new volume.

[dfoxg]

@derekbit do you have a timeline for this PR?

[dfoxg]

@dchirikov for a quick win, would it be enough to remove the nodeAffinity on already created pv´s?

EDIT: nope didn´t work, field is immutable :(

[dimon222]

Is there a viable alternative? Anything that can be just installed as extra storage class by any chance?

[plevart]

This branch works for me. I have a GFS2 cluster on the k8s nodes and ReadWriteMany is very suitable for that. For example, to enable single-instance database such as MySQL to not be "pinned" to single node. Of course you have to be careful to avoid corrupting the database when failing-over an instance from one node to another as k8s does not "fence" one instance from another, but databases usually contain an option that prevents such concurrent access or coordinates it from multiple instances (external locking for MySQL for example). It would be nice to integrate this to main. Are there any reservations? Potential problems?

[dchirikov]

Hey guys. Any chance this PR will be merged? It being open for more than a year already.

[plevart]

@dchirikov perhaps, If you could rebase it to master (it currently has conflicts), it would be easier for maintainers to merge it. I too am interested for this to get merged. Currently I use your branch, but master has progressed and new versions were released in the meantime...

[derekbit]

Hey guys. Any chance this PR will be merged? It being open for more than a year already.

Yeah, I will review again and merge it after rebase. Thank you.

[dfoxg]

Hi @derekbit are there any updates?

[derekbit]

@dfoxg
Sorry for the late review. I will finish the review by next Friday.

[derekbit]

@dchirikov In general, LGTM. Could you please add an example (examples). A test (examples) is also appreciated.

[dchirikov]

@derekbit Hi there! I resolved conflicts and added example. Please have a look.

[Vinrobot]

Hi, really interested in seeing this being merged, but I was testing the Helm chart and I think that the following file was forgotten!

https://github.com/dchirikov/local-path-provisioner/blob/shared_fs/deploy/chart/local-path-provisioner/templates/configmap.yaml#L10

And thanks for the work @dchirikov!