disabling pv affinity

[ahmadalli]

I'm using local-path-provisioner with a directory backed by gluster which is shared through all my nodes. but I can't have multiple pods with shared pvc on multiple nodes because my the pv has nodeAffinity

Is it possible to disable the node affinity of the pv?

[yasker]

@ahmadalli The local path provisioner is designed for using local disk for volumes, so they're configured to always affiliate with the node.

Since you're using glusterfs, it's better for you to use a Kubernetes driver supports glusterfs.

[ahmadalli]

@yasker isn't it possible to make nodeAffinity optional? that would solve my problem

[yasker]

@ahmadalli I don't think it would be the ideal way to solve your issue. What if you mount the glusterfs into different directories on the different nodes? Local path provisioner wouldn't have a way to know that.

If you just want to use the same path on every host, you probably can use the hostpath volume directly.

[ahmadalli]

@yasker I have an ansible playbook for mounting gluster volume so the directory would be mounted on the same directory. I like to utilize the features of pvcs and have the option to use other storage classes. what I'm saying is it's good to have the option.

one other use case of this feature is the ability to move some pod with it's data stored in a directory to another node just by copying data from node a to node b and then changing nodeSelector of the deployment and it would be done

[WhiteBahamut]

I'd also think it should be an option which the user can turn off. Default should turned on. If I want to turn it off, I am reponsible to make sure the local path exists on all my nodes.

[rafadesu]

I support this too. It was driving me crazy that all the replicas in my deployment were being assigned to the same node until I discovered that the PV has node affinity set to a particular node. In my case, I have the same local path mounted to same NFS volume therefore node affinity is not really required in the PV.

It would be good to allow for a configuration to remove it so I don't have to install an NFS provisioner just for this.

The code to make the node affinity section optional should be trivial and powerful addition to this provisioner.

Also, I've seen another request (https://github.com/rancher/local-path-provisioner/issues/186) to use a node label instead of kubernetes.io/hostname... maybe that solution could also help here too by assigning the same label to all the nodes sharing the same NFS, GlusterFS,... mount.

[rafadesu]

In case some people are interested, I've forked the repo and did a modification to use labels instead of node affinity and allow ReadWriteMany so this can be used as described here with mounted shared paths. I'm not very good with go so I didn't yet did it so it is configurable. But it might be useful for people that need this modification. Check this fork: https://github.com/rafadesu/local-path-provisioner

[Kribylet]

This issue is relevant to my setup where I use NFS to back a shared storage path for multiple nodes. When a node fails I would like pods to be scheduled to replace those on the failed node. Mounting storage using hostPath causes SecurityContext permission issues which seems to require reckless chmod behavior to fix since kubelet handles permissions for hostPath.

I would like to be able to provide a nodeAffinity for persistentVolumes created by the provisioner which, if and only if provided replace the default nodeAffinity. Ideally I would be able to label nodes according to their storage configuration and then provide nodeAffinity configurations for the storage provisioner based on those node labels. Failure to provide a valid nodeAffinity configuration feels like a user error to me, especially when omitting said configuration results in a working configuration.

[kervel]

Another usecase: per-node disk cache.

i have a workload that does an expensive computation and stores the results in a cache folder. The cache is then used by the same pod. For the cache to be performant, it needs to be local (it is a lot of data). A local-path volume would be ideal: without affinity, when the workloads gets scheduled on a new node, the cache would be empty on that node, which would result in the computation to be triggered. If the cache is already filled and up to date, it would simply be used.