terraform-hcloud-k3s
terraform-hcloud-k3s copied to clipboard
cicdteam
Join over private ip
Hi, why is it necessary to pass the public IP of the master server?
- https://github.com/cicdteam/terraform-hcloud-k3s/blob/main/modules/node_group/templates/init.sh#L10
Indeed, ideally, the private IP should be used. Also, ideally, we would not have a network overlay with flannel. We can use cilium's native routing over the private network, as the Hetzner team does in their tests here https://github.com/hetznercloud/hcloud-cloud-controller-manager/tree/master/e2etests
The values don't seem to know about the server IP, it can be fetched after spinning up: hcloud server describe ${SE_DOMAIN}-master | grep -A5 "Private Net:" | grep -o "[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+"
That not possible to feed back into provisioning?
How would the overlay fix the issue?
@systemstart Not sure I understand your question to the fullest, but definitely the CCM's job should be to bring all of that info into the cluster if I'm not mistaken. You shouldn't need to fetch the IPs manually at all.
The original question was, how to use the private network for Advertise/Join.
You suggested flannel or some Go source files as possible solutions for this.
Would you go a bit more into details?
Fixed in multiple forks, e.g. https://github.com/identinet/terraform-hcloud-k3s/commit/4edbb4a4ce646a184b12e42dfca4c5ac8f39064d
Indeed, ideally, the private IP should be used. Also, ideally, we would not have a network overlay with flannel. We can use cilium's native routing over the private network, as the Hetzner team does in their tests here https://github.com/hetznercloud/hcloud-cloud-controller-manager/tree/master/e2etests
@mysticaltech Not quite sure what you mean by that and how to use the native network. Can you point me to an example? I replaced flannel by calico, maybe this is already helpful to you: https://github.com/identinet/terraform-hcloud-k3s/commit/e98776ba62ae353d5b78b7ba24759a26b5fb2af4
These pieces of documentation were most helpful:
- https://github.com/hetznercloud/hcloud-cloud-controller-manager#networks-support -> and then go to the advanced documentation ->
- https://github.com/hetznercloud/hcloud-cloud-controller-manager/blob/master/docs/deploy_with_networks.md
Most of the configuration was already in place.
Thanks, I've moved on too. I just make sure to configure the right interface to use by flannel, the eth1 with a Hetzner private network. It gives me what I needed.