Update of the StaticEgressIP object does not work properly

[uablrek]

Various updates of the StaticEgressIP object does not work properly.

Remove the StaticEgressIP object

Setup a staticegressip then delete it with;

kubectl delete staticegressip egressip-alpine

The egress-ip function is removed, outgoing connects from the pod are again NAT'ed to the node ip.

On the directors everything seems ok; The ipset and the rule in the mangle table on the directors are correctly removed.

But on the gateway some settings are not cleaned-up;

The SNAT rule is not removed;

Chain STATIC-EGRESS-NAT-CHAIN (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    1    60 SNAT       all  --  *      *       0.0.0.0/0            192.168.2.0/24       match-set EGRESS-IP-QPAZYHZ2OUEYTPUQ src to:15.0.0.1

Also the ipset is not removed, but all entries are flushed;

# ipset list EGRESS-IP-QPAZYHZ2OUEYTPUQ
Name: EGRESS-IP-QPAZYHZ2OUEYTPUQ
Type: hash:ip
Revision: 4
Header: family inet hashsize 1024 maxelem 65536 timeout 0
Size in memory: 88
References: 2
Number of entries: 0

Update the egressip

The egressip in an existing StaticEgressIP object is updated from "15.0.0.1" to "15.0.0.13".

On the directors nothing should be altered, and it isn't.

On the gateway the old SNAT rule is not removed which I guess is the same issue as described above for removal of the object. The new SNAT rule is fortunately inserted before the old one so it seem to take precedence and the egress-ip is SNAT'ed correctly.

Chain STATIC-EGRESS-NAT-CHAIN (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    1    60 SNAT       all  --  *      *       0.0.0.0/0            192.168.2.0/24       match-set EGRESS-IP-QPAZYHZ2OUEYTPUQ src to:15.0.0.13
    0     0 SNAT       all  --  *      *       0.0.0.0/0            192.168.2.0/24       match-set EGRESS-IP-QPAZYHZ2OUEYTPUQ src to:15.0.0.1

Update the cidr

The cidr in the StaticEgressIP object is updated from "192.168.2.0/24" to "111.0.0.0/24".

Connects to the new cidr are correctly SNAT'ed to the egress-ip, but connects to the old cidr are still (incorrectly) also SNAT'ed.

On directors the new cidr is added but the old one is not removed;

# ip ro show table kube-static-egress-ip
111.0.0.0/24 via 192.168.1.3 dev eth1 
192.168.2.0/24 via 192.168.1.3 dev eth1 

On the gateway the old SNAT rule remains;

    1    60 SNAT       all  --  *      *       0.0.0.0/0            111.0.0.0/24         match-set EGRESS-IP-QPAZYHZ2OUEYTPUQ src to:15.0.0.1
    2   120 SNAT       all  --  *      *       0.0.0.0/0            192.168.2.0/24       match-set EGRESS-IP-QPAZYHZ2OUEYTPUQ src to:15.0.0.1