local-path-provisioner icon indicating copy to clipboard operation
local-path-provisioner copied to clipboard

Will local-path-provisioner separate pvc replicas across the cluster

Open sadekunle opened this issue 4 years ago • 5 comments

I'm deploying a statefulset workload with 3 replicas. I see that the local-path-provisioner deploys all pvc replicas to one node. Is there a way to separate the pvc for each replica to a different node?

sadekunle avatar Aug 10 '20 17:08 sadekunle

The scheduling of the storage is not controlled by the local path provisioner. It will waitForFirstConsumer (a.k.a. pod creation/scheduling). So if you use anti-affinity for the pods during the first creation, the volumes should follow the pod's scheduling to the different nodes.

After first time, the PV would bound to a certain node and cannot be moved after.

yasker avatar Aug 10 '20 20:08 yasker

Would I need to create a pvc for each replica? Trying to deploy the 3 replicas and after the first replica is deployed I get the following error.

Warning FailedScheduling default-scheduler 0/4 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity, 1 node(s) didn't satisfy existing pods anti-affinity rules, 3 node(s) had volume node affinity conflict.

sadekunle avatar Aug 11 '20 01:08 sadekunle

If there is only one PVC, it will only able to be on one node, since the data is stored in one node's hard drive.

yasker avatar Aug 11 '20 16:08 yasker

Statefulset and volumeClaimTemplates may fit your needs

see https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/

ThinCats avatar Aug 13 '20 15:08 ThinCats

I pointed my local-path-provisioner-Path to an glusterfs-mount, so all of my files are available on all k8s-nodes. How can I configure the local-path-provider that the error had volume node affinity conflict does not appear?

dfoxg avatar Feb 02 '22 09:02 dfoxg