Nodeport timeout when remote node in different subnet (IPv6-only cluster)

Open sjainbd opened this issue 1 year ago • 0 comments

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?

Requests are timing out, from world to a node IP's NodePort, only when the backend pod is scheduled on a node in a different subnet, under this scenario:

native-routing mode,
in an IPv6-only cluster,
with kube-proxy replacement in SNAT mode
with the NodePort Service externalTrafficPolicy and internalTrafficPolicy set to Cluster

I've followed the native routing docs (https://docs.cilium.io/en/stable/network/concepts/routing/#native-routing) which suggest using kube-router to run BGP to distribute the routes (https://docs.cilium.io/en/stable/network/kube-router/).

However, these requests succeed:

from world to a nodeA that is hosting the backend pod
from world to a nodeB that is on the same subnet as the node hosting the backend pod
from a nodeC in a different subnet itself to localhost:nodeport

It is just requests from world to [public IP]:nodeport of a node on a different subnet that fail (timeout).

I tried doing a tcpdump comparing the behaviour in the 3 above flows. Here's a diagram to explain the issue I'm seeing based on those dumps:

cilium-nodeport-diagram drawio (2)

Specifically, in the failure case, if I tcpdump on NodeC, I can see:

The initial incoming request from World->NodeC
An outgoing request from NodeC->Pod
An incoming request from Pod->NodeC
No final response from NodeC->World

I think this issue may be related to the IPIP tunnel routes that are installed by kube-router for all nodes on a different subnet (this behaviour is stated in the linked doc too). The main difference I can see is that in the failure case, the requests from NodeC->Pod and Pod->NodeC go through one of the tun* interfaces, whereas in the success case, everything always goes through eth0. So could it be that when the Pod response comes in on tun*, Cilium doesn't actually see the response?

I do not see any drops in cilium monitor or hubble observe.

I think I've exhausted all the different combinations of flags to test if this is a misconfiguration but I can't get this to work with any of the following changes:

--enable-host-legacy-routing=true (no difference)
--bpf-lb-mode=dsr (In DSR mode, I actually got a bit closer -- the final response does reach the world node, but the source IP is of the pod, so it looks like it is dropped. But this seems like a different issue, so I could post a separate issue if required)
--direct-routing-device=eth0
--devices=eth0,tun+ (no difference, but this should be expected to not make a difference)
--enable-bpf-masquerade=true (no difference, but this should be expected to not make a difference)
--direct-routing-skip-unreachable=true along with --auto-direct-node-routes=true (breaks networking)
I also tried adding CILIUM_FORWARD ip6tables rules for ingress tun+, similar to the existing -A CILIUM_FORWARD -i lxc+ -m comment --comment "cilium: cluster->any on lxc+ forward accept" -j ACCEPT, but no difference (but nodeports are managed by BPF AFAIK, so I think this is expected)

Various other things I've checked following the troubleshooting guide (https://docs.cilium.io/en/stable/operations/troubleshooting/):

cilium status --verbose shows Cluster health as all reachable, and all nodes and endpoints are reachable
cilium-health status, everything is OK
No suspicious agent logs
cilium service list shows all the NodePorts correctly

Cilium Version

Client: 1.16.0-rc.0 14c2b3d0 2024-06-17T16:32:13+02:00 go version go1.22.4 linux/amd64
Daemon: 1.16.0-rc.0 14c2b3d0 2024-06-17T16:32:13+02:00 go version go1.22.4 linux/amd64

Kernel Version

Linux host01 5.15.56.bsk.1-amd64 #5.15.56.bsk.1 SMP Debian 5.15.56.bsk.1 Mon Jul 25 03:31:05 UTC  x86_64 x86_64 x86_64 GNU/Linux

Kubernetes Version

Client Version: v1.28.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.2

Regression

N/A

Sysdump

No response

Relevant log output

time="2024-06-28T09:52:47Z" level=info msg="  --agent-health-port='9879'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --agent-labels=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --agent-liveness-update-interval='1s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --agent-not-ready-taint-key='node.cilium.io/agent-not-ready'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --allocator-list-timeout='3m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --allow-icmp-frag-needed='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --allow-localhost='auto'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --annotate-k8s-node='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --api-rate-limit=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --arping-refresh-period='30s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --auto-create-cilium-node-resource='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --auto-direct-node-routes='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bgp-announce-lb-ip='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bgp-announce-pod-cidr='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bgp-config-path='/var/lib/cilium/bgp/config.yaml'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-auth-map-max='524288'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-ct-global-any-max='262144'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-ct-global-tcp-max='524288'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-ct-timeout-regular-any='1m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-ct-timeout-regular-tcp='2h13m20s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-ct-timeout-regular-tcp-fin='10s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-ct-timeout-regular-tcp-syn='1m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-ct-timeout-service-any='1m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-ct-timeout-service-tcp='2h13m20s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-ct-timeout-service-tcp-grace='1m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-events-drop-enabled='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-events-policy-verdict-enabled='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-events-trace-enabled='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-filter-priority='1'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-fragments-map-max='8192'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-acceleration='disabled'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-affinity-map-max='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-algorithm='random'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-dsr-dispatch='opt'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-dsr-l4-xlate='frontend'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-external-clusterip='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-maglev-hash-seed='JLfvgnHc2kaSUFaI'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-maglev-map-max='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-maglev-table-size='16381'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-map-max='65536'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-mode='snat'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-rev-nat-map-max='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-rss-ipv4-src-cidr=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-rss-ipv6-src-cidr=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-service-backend-map-max='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-service-map-max='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-sock='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-sock-hostns-only='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-lb-source-range-map-max='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-map-dynamic-size-ratio='0.0025'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-map-event-buffers=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-nat-global-max='524288'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-neigh-global-max='524288'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-node-map-max='16384'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-policy-map-full-reconciliation-interval='15m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-policy-map-max='16384'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-root='/sys/fs/bpf'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bpf-sock-rev-map-max='262144'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --bypass-ip-availability-upon-restore='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --certificates-directory='/var/run/cilium/certs'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cflags=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cgroup-root='/run/cilium/cgroupv2'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cilium-endpoint-gc-interval='5m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cluster-health-port='4240'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cluster-id='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cluster-name='default'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cluster-pool-ipv4-cidr='10.0.0.0/8'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cluster-pool-ipv4-mask-size='24'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cluster-pool-ipv6-cidr='2001:db8:42:0::/51'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cluster-pool-ipv6-mask-size='120'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --clustermesh-config='/var/lib/cilium/clustermesh/'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --clustermesh-ip-identities-sync-timeout='1m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --clustermesh-sync-timeout='1m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cmdref=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cni-chaining-mode='none'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cni-chaining-target=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cni-exclusive='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cni-external-routing='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cni-log-file='/var/run/cilium/cilium-cni.log'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --cnp-node-status-gc-interval='0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --config=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --config-dir='/tmp/cilium/config-map'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --config-sources='config-map:kube-system/cilium-config'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --conntrack-gc-interval='0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --conntrack-gc-max-interval='0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --container-ip-local-reserved-ports='auto'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --controller-group-metrics=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --crd-wait-timeout='5m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --custom-cni-conf='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --datapath-mode='veth'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --debug='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --debug-verbose='flow,kvstore,envoy,datapath,policy'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --derive-masq-ip-addr-from-device=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --devices=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --direct-routing-device=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --direct-routing-skip-unreachable='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --disable-cnp-status-updates='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --disable-endpoint-crd='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --disable-envoy-version-check='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --disable-external-ip-mitigation='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --disable-iptables-feeder-rules=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --dns-max-ips-per-restored-rule='1000'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --dns-policy-unload-on-shutdown='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --dnsproxy-concurrency-limit='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --dnsproxy-concurrency-processing-grace-period='0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --dnsproxy-enable-transparent-mode='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --dnsproxy-lock-count='131'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --dnsproxy-lock-timeout='500ms'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --egress-gateway-policy-map-max='16384'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --egress-gateway-reconciliation-trigger-interval='1s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --egress-masquerade-interfaces=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --egress-multi-home-ip-rule-compat='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-active-connection-tracking='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-auto-protect-node-port-range='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-bandwidth-manager='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-bbr='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-bgp-control-plane='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-bpf-clock-probe='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-bpf-masquerade='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-bpf-tproxy='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-cilium-api-server-access='*'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-cilium-endpoint-slice='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-cilium-health-api-server-access='*'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-custom-calls='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-encryption-strict-mode='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-endpoint-health-checking='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-endpoint-routes='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-envoy-config='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-external-ips='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-gateway-api='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-health-check-loadbalancer-ip='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-health-check-nodeport='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-health-checking='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-high-scale-ipcache='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-host-firewall='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-host-legacy-routing='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-host-port='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-hubble='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-hubble-recorder-api='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-icmp-rules='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-identity-mark='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ingress-controller='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ip-masq-agent='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipip-termination='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipsec='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipsec-encrypted-overlay='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipsec-key-watcher='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipsec-xfrm-state-caching='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipv4='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipv4-big-tcp='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipv4-egress-gateway='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipv4-fragment-tracking='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipv4-masquerade='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipv6='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipv6-big-tcp='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipv6-masquerade='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-ipv6-ndp='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-k8s='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-k8s-api-discovery='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-k8s-endpoint-slice='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-k8s-networkpolicy='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-k8s-terminating-endpoint='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-l2-announcements='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-l2-neigh-discovery='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-l2-pod-announcements='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-l7-proxy='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-local-node-route='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-local-redirect-policy='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-masquerade-to-route-source='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-mke='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-monitor='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-nat46x64-gateway='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-node-port='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-node-selector-labels='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-pmtu-discovery='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-policy='default'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-recorder='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-remote-node-identity='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-runtime-device-detection='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-sctp='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-service-topology='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-session-affinity='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-srv6='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-stale-cilium-endpoint-cleanup='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-svc-source-range-check='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-tcx='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-tracing='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-unreachable-routes='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-vtep='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-well-known-identities='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-wireguard='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-wireguard-userspace-fallback='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-xdp-prefilter='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --enable-xt-socket-fallback='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --encrypt-interface=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --encrypt-node='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --encryption-strict-mode-allow-remote-node-identities='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --encryption-strict-mode-cidr=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --endpoint-bpf-prog-watchdog-interval='30s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --endpoint-gc-interval='5m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --endpoint-queue-size='25'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --envoy-base-id='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --envoy-config-retry-interval='15s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --envoy-config-timeout='2m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --envoy-keep-cap-netbindservice='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --envoy-log=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --envoy-secrets-namespace=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --exclude-local-address=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --exclude-node-label-patterns=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --external-envoy-proxy='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --fixed-identity-mapping=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --force-device-detection='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --fqdn-regex-compile-lru-size='1024'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --gateway-api-secrets-namespace=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --gops-port='9890'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --http-idle-timeout='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --http-max-grpc-timeout='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --http-normalize-path='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --http-request-timeout='3600'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --http-retry-count='3'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --http-retry-timeout='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-disable-tls='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-drop-events='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-drop-events-interval='2m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-drop-events-reasons='auth_required,policy_denied'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-event-buffer-capacity='4095'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-event-queue-size='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-export-allowlist=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-export-denylist=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-export-fieldmask=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-export-file-compress='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-export-file-max-backups='5'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-export-file-max-size-mb='10'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-export-file-path=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-flowlogs-config-path=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-listen-address=':4244'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-metrics=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-metrics-server=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-monitor-events=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-prefer-ipv6='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-recorder-sink-queue-size='1024'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-recorder-storage-path='/var/run/cilium/pcaps'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-redact-enabled='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-redact-http-headers-allow=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-redact-http-headers-deny=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-redact-http-urlquery='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-redact-http-userinfo='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-redact-kafka-apikey='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-skip-unknown-cgroup-ids='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-socket-path='/var/run/cilium/hubble.sock'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-tls-cert-file='/var/lib/cilium/tls/hubble/server.crt'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-tls-client-ca-files='/var/lib/cilium/tls/hubble/client-ca.crt'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --hubble-tls-key-file='/var/lib/cilium/tls/hubble/server.key'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --identity-allocation-mode='crd'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --identity-change-grace-period='5s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --identity-gc-interval='15m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --identity-heartbeat-timeout='30m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --identity-restore-grace-period='30s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ingress-secrets-namespace=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --install-iptables-rules='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --install-no-conntrack-iptables-rules='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ip-masq-agent-config-path='/etc/config/ip-masq-agent'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipam='kubernetes'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipam-cilium-node-update-rate='15s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipam-default-ip-pool='default'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipam-multi-pool-pre-allocation=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipsec-key-file=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipsec-key-rotation-duration='5m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --iptables-lock-timeout='5s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --iptables-random-fully='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv4-native-routing-cidr=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv4-node='auto'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv4-pod-subnets=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv4-range='auto'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv4-service-loopback-address='169.254.42.1'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv4-service-range='auto'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv6-cluster-alloc-cidr='f00d::/64'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv6-mcast-device=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv6-native-routing-cidr='2001:db8:42:0::/51'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv6-node='auto'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv6-pod-subnets=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv6-range='auto'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --ipv6-service-range='auto'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --join-cluster='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-api-server=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-client-burst='10'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-client-connection-keep-alive='30s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-client-connection-timeout='30s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-client-qps='5'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-heartbeat-timeout='30s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-kubeconfig-path=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-namespace='kube-system'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-require-ipv4-pod-cidr='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-require-ipv6-pod-cidr='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-service-cache-size='128'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-service-proxy-name=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-sync-timeout='3m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --k8s-watcher-endpoint-selector='metadata.name!=kube-scheduler,metadata.name!=kube-controller-manager,metadata.name!=etcd-operator,metadata.name!=gcp-controller-manager'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --keep-config='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --kube-proxy-replacement='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --kube-proxy-replacement-healthz-bind-address=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --kvstore=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --kvstore-connectivity-timeout='2m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --kvstore-lease-ttl='15m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --kvstore-max-consecutive-quorum-errors='2'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --kvstore-opt=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --kvstore-periodic-sync='5m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --l2-announcements-lease-duration='15s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --l2-announcements-renew-deadline='5s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --l2-announcements-retry-period='2s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --l2-pod-announcements-interface=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --label-prefix-file=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --labels=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --lib-dir='/var/lib/cilium'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --local-max-addr-scope='252'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --local-router-ipv4=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --local-router-ipv6=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --log-driver=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --log-opt=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --log-system-load='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --max-connected-clusters='255'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --max-controller-interval='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --max-internal-timer-delay='0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mesh-auth-enabled='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mesh-auth-gc-interval='5m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mesh-auth-mutual-connect-timeout='5s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mesh-auth-mutual-listener-port='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mesh-auth-queue-size='1024'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mesh-auth-rotated-identities-queue-size='1024'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mesh-auth-signal-backoff-duration='1s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mesh-auth-spiffe-trust-domain='spiffe.cilium'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mesh-auth-spire-admin-socket=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --metrics=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mke-cgroup-mount=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --monitor-aggregation='none'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --monitor-aggregation-flags='all'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --monitor-aggregation-interval='5s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --monitor-queue-size='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --mtu='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --multicast-enabled='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --nat-map-stats-entries='32'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --nat-map-stats-interval='30s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --node-encryption-opt-out-labels='node-role.kubernetes.io/control-plane'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --node-labels=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --node-port-acceleration='disabled'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --node-port-algorithm='random'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --node-port-bind-protection='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --node-port-mode='snat'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --node-port-range='30000,32767'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --nodeport-addresses=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --nodes-gc-interval='5m0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --operator-api-serve-addr='[::1]:9234'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --policy-accounting='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --policy-audit-mode='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --policy-cidr-match-mode=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --policy-queue-size='100'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --policy-trigger-interval='1s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --pprof='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --pprof-address='localhost'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --pprof-port='6060'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --preallocate-bpf-maps='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --prepend-iptables-chains='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --procfs='/host/proc'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --prometheus-serve-addr=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-admin-port='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-connect-timeout='2'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-gid='1337'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-idle-timeout-seconds='60'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-max-connection-duration-seconds='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-max-requests-per-connection='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-portrange-max='20000'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-portrange-min='10000'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-prometheus-port='9964'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-xff-num-trusted-hops-egress='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --proxy-xff-num-trusted-hops-ingress='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --read-cni-conf=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --remove-cilium-node-taints='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --restore='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --route-metric='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --routing-mode='native'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --service-no-backend-response='reject'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --set-cilium-is-up-condition='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --set-cilium-node-taints='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --sidecar-istio-proxy-image='cilium/istio_proxy'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --skip-cnp-status-startup-clean='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --socket-path='/var/run/cilium/cilium.sock'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --srv6-encap-mode='reduced'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --state-dir='/var/run/cilium'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --static-cnp-path=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --synchronize-k8s-nodes='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tofqdns-dns-reject-response-code='nameError'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tofqdns-enable-dns-compression='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tofqdns-endpoint-max-ip-per-hostname='50'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tofqdns-idle-connection-grace-period='0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tofqdns-max-deferred-connection-deletes='10000'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tofqdns-min-ttl='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tofqdns-pre-cache=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tofqdns-proxy-port='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tofqdns-proxy-response-max-delay='100ms'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --trace-payloadlen='128'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --trace-sock='true'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tunnel-port='0'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --tunnel-protocol='vxlan'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --unmanaged-pod-watcher-interval='15'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --use-cilium-internal-ip-for-ipsec='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --use-full-tls-context='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --version='false'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --vlan-bpf-bypass=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --vtep-cidr=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --vtep-endpoint=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --vtep-mac=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --vtep-mask=''" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --wireguard-persistent-keepalive='0s'" subsys=daemon
time="2024-06-28T09:52:47Z" level=info msg="  --write-cni-conf-when-ready='/host/etc/cni/net.d/05-cilium.conflist'" subsys=daemon

Anything else?

References: https://docs.cilium.io/en/stable/network/concepts/routing/#native-routing, https://docs.cilium.io/en/stable/network/kube-router/, https://github.com/cloudnativelabs/kube-router/blob/master/docs/user-guide.md.

Cilium Users Document

[ ] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Jun 28 '24 10:06 sjainbd