hetznercloud
What is proper procedure to re-attach volume from one node to another
Hello,
I haven't been able to find answer to that question, so I hope I haven't overlooked anything. I found #44, but I'm not sure if it's my use case.
My use case scenario: I would like to move my pod (attached to hetzner cloud volume via PVC) from one node to another. I'm not sure how.
- If I forced pod to run on another node, my deployment was in loop trying to create new pod and failing with outofmemory state. In events, I got information about node affinity for volume not being matched (MountVolume.NodeAffinity check failed for volume "pvc-cb390a70-b74f-4e08-bbd2-57f3acc3c0b8" : no matching NodeSelectorTerms)
- If I reattach volume from one node to another (in cloud console), I see that volume is in attach / reattach loop at cloud console.
- I tried to change annotation in persistent volume claim definition volume.kubernetes.io/selected-node - no luck here.
Is there a way to do so?
@jsobiecki You'll find out that this driver isn't very good I've been battling with it for months - sometimes volumes are moved with pods immediately, sometimes they are never moved I resorted to tainting my nodes as noExecute to force pods to move, which they did, and the volumes were de and re attached where they were needed SOME volumes unfortunately, that's my experience so far I'd say don't do it yourself, as it'll cause issues, try force the pod with the volume to move to another node your results may vary, though
@jsobiecki Since my last reply, I've discovered a couple of things on rancher distribution, rke2 or k3s, this just won't work nicely Will work once, and then if you never touch it, you are fine, but any change and you're gonna have it blow in your face What k8s flavour are you using?
@jsobiecki yeah, try something else - I got it working nicely with kubeone haven;t had major issues since
Not sure why, but I see that this is working now correctly. After pod recreation at different node, volume has been re-attached there.