helm-charts icon indicating copy to clipboard operation
helm-charts copied to clipboard

[Feature/Idea] Add/Adjust node label(s) on what node has the VIP active

Open soakes opened this issue 6 years ago • 4 comments

Just an idea, it would be good if you could add say a label called vip=active or something on any node which has the VIP active.

But the issue is, no access to kubectl and even if that's on the image, that's not really very secure and I don't know k8s well enough yet to find an alternative solution. So if anyone got any ideas, please let me know.

Without this feature, you can't like bring up a service which has host-based networking which can follow around the VIP.

i.e. changing the script to something like this:

    #!/bin/bash
    echo "Executing vip-notify script for Hetzner Cloud"
    ENDSTATE=$3
    NAME=$2
    TYPE=$1
    if [ "$ENDSTATE" == "MASTER" ] ; then
        HOST_IP=$(ip route get 8.8.8.8 | awk '{print $7 }')
        ACTIVE_HOST=$(/bin/hostname -s)
        export SERVER_ID=$(curl -s -H "Authorization: Bearer $HETZNER_TOKEN" "https://api.hetzner.cloud/v1/servers?name=$HOSTNAME" | grep -C 2 servers | grep id | awk '{ print $2 }' | sed -e s/,//)
        export FLOATING_IP_ID=$(curl -s -H "Authorization: Bearer $HETZNER_TOKEN" "https://api.hetzner.cloud/v1/floating_ips" | grep "\"ip\": \"$FLOATING_IP" -B 3  | grep id | awk '{ print $2 }' | sed -e s/,//)
        echo "Installing dnsutils"
        apt update && apt install -y dnsutils
        echo "Setting Failover IP (Floating IP): $FLOATING_IP to Server IP:  $HOST_IP"
        curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $HETZNER_TOKEN" -d "{\"server\":$SERVER_ID}" "https://api.hetzner.cloud/v1/floating_ips/$FLOATING_IP_ID/actions/assign"
        echo "Clearing VIP label from all nodes"
        kubectl get nodes --show-labels|grep vip|awk '{print $1}'|while read i; do kubectl label nodes $i vip-;done
        echo "Setting VIP label on ACTIVE node"
        kubectl label nodes "${ACTIVE_HOST}" vip=active
    fi

Update Found away without using kubectl

curl -k -v -H "Accept: application/json" -XPATCH -d '{"metadata":{"labels":{"vip":"active"}}}' -H "Content-Type: application/merge-patch+json" http://127.0.0.1:8080/api/v1/nodes/node01-k8s

-Working on some updates will post when done.-

Update This is quick and dirty, but this does work.

$ cat configmap-script.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: vip-notify
data:
  notify.sh: |
    #!/bin/bash
    echo "Executing vip-notify script for Hetzner Cloud"
    ENDSTATE=$3
    NAME=$2
    TYPE=$1
    if [ "$ENDSTATE" == "MASTER" ] ; then
        HOST_IP=$(ip route get 8.8.8.8 | awk '{print $7 }')
        SHOST=$(echo $HOSTNAME|awk -F. '{print $1}')
        export SERVER_ID=$(curl -s -H "Authorization: Bearer $HETZNER_TOKEN" "https://api.hetzner.cloud/v1/servers?name=$HOSTNAME" | grep -C 2 servers | grep id | awk '{ print $2 }' | sed -e s/,//)
        export FLOATING_IP_ID=$(curl -s -H "Authorization: Bearer $HETZNER_TOKEN" "https://api.hetzner.cloud/v1/floating_ips" | grep "\"ip\": \"$FLOATING_IP" -B 3  | grep id | awk '{ print $2 }' | sed -e s/,//)
        echo "Setting Failover IP (Floating IP): $FLOATING_IP to Server IP:  $HOST_IP"
        curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $HETZNER_TOKEN" -d "{\"server\":$SERVER_ID}" "https://api.hetzner.cloud/v1/floating_ips/$FLOATING_IP_ID/actions/assign"
        echo "Configure VIP label on all nodes to false"
        curl -s http://127.0.0.1:8080/api/v1/nodes|grep selfLink|awk -F"/" '{print $5}' | awk -F\" '{print $1}' |sed 1d|while read i; do curl -k -v -H "Accept: application/json" -XPATCH -d '{"metadata":{"labels":{"vip":"false"}}}' -H "Content-Type: application/merge-patch+json" http://127.0.0.1:8080/api/v1/nodes/$i;done
        echo "Setting VIP label on ACTIVE node to true"
        curl -k -v -H "Accept: application/json" -XPATCH -d '{"metadata":{"labels":{"vip":"true"}}}' -H "Content-Type: application/merge-patch+json" http://127.0.0.1:8080/api/v1/nodes/"${SHOST}"
    fi

This sets vip=false to every node and then sets just vip=true on the node which has the VIP.

I am sure there's a better way to do it but currently, this is the only way ive found, while not elegant, it does work for now.

soakes avatar Jun 10 '18 19:06 soakes

Glad to hear 👂🏾. When I have time I will add support for multiple IPs too.

exocode avatar Jun 12 '18 14:06 exocode

Hi there,

I should have some time at the weekend to look further into it but anything you can come up with would be great, always can tweak and send the tweaks to you. I've been going through the code which you have created and I think I see how it's done. I have made a few minor tweaks to the script atm and will post a revised one soon. This just adds an extra-label right now to the nodes so you can tie it to containers easily (i.e. containers follow VIP around). As for the extra IP, not sure yet how that's done, the bash side I can see but the k8s I don't (currently).

I must admit I am just starting on the k8s path myself too, its that side I don't yet fully understand but have been a docker swarm user for a long time now, so docker itself isn't a problem, it's just the k8s way that is.

Kind Regards.

soakes avatar Jun 13 '18 05:06 soakes

Maybe we could "loop" through a "key/value pair array" like fips="123.456.789.0=www.domain.com,11aa::11aa::11aa:11aa=another.domain.com. These could include ipv6 as well as ipv4 domains. Or we split it like fipv4 and fipv6 parameter. And write them into the configmap. Cornelius had made a PR on the notify script repo to be able to use the official one.

exocode avatar Jun 13 '18 07:06 exocode

Nice idea.

soakes avatar Jun 13 '18 07:06 soakes