flannel
flannel copied to clipboard
Published
20 hours ago •
flannel-io
flannel-io
Cannot create SubnetManager: net/http: TLS handshake timeout - CrashBackLoopOff
Situation
Hi I have a node behind NAT and to fix that I set up a wireguard VPN between all my nodes. But the kube flannel pod running on that under NAT node is in CrashBackLoopOff state. All the other nodes are running perfectly. Here are the logs of the kube-flannel pod:
Logs
I0718 15:10:31.416192 1 main.go:207] CLI flags config: {etcdEndpoints:http://127.0.0.1:4001,http://127.0.0.1:2379 etcdPrefix:/coreos.com/network etcdKeyfile: etcdCertfile: etcdCAFile: etcdUsername: etcdPassword: version:false kubeSubnetMgr:true kubeApiUrl: kubeAnnotationPrefix:flannel.alpha.coreos.com kubeConfigFile: iface:[] ifaceRegex:[] ipMasq:true ifaceCanReach: subnetFile:/run/flannel/subnet.env publicIP: publicIPv6: subnetLeaseRenewMargin:60 healthzIP:0.0.0.0 healthzPort:0 iptablesResyncSeconds:5 iptablesForwardRules:true netConfPath:/etc/kube-flannel/net-conf.json setNodeNetworkUnavailable:true}
W0718 15:10:31.512917 1 client_config.go:614] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
E0718 15:10:41.534810 1 main.go:224] Failed to create SubnetManager: error retrieving pod spec for 'kube-flannel/kube-flannel-ds-7jq79': Get "https://10.96.0.1:443/api/v1/namespaces/kube-flannel/pods/kube-flannel-ds-7jq79": net/http: TLS handshake timeout
Explanation
I tried to curl 10.96.0.1:443, sometimes it works and sometimes it doesnt. This is the problem. I can always ping 10.66.66.1:6443 (master ip) but the 10.96.0.1 seems to have something weird. I tried to loop at the iptables but nothing was found:
Iptables
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N KUBE-KUBELET-CANARY
-N KUBE-MARK-DROP
-N KUBE-MARK-MASQ
-N KUBE-NODEPORTS
-N KUBE-POSTROUTING
-N KUBE-PROXY-CANARY
-N KUBE-SEP-6E7XQMQ4RAYOWTTM
-N KUBE-SEP-HHJOKZF3Y7Y3YORK
-N KUBE-SEP-IT2ZTR26TO4XFPTO
-N KUBE-SEP-LV3ODC7YAWYMJH2A
-N KUBE-SEP-N4G2XR5TDX7PQE7P
-N KUBE-SEP-SFKEIAIVHDK3MCWO
-N KUBE-SEP-V7IHSQL4ET5F356W
-N KUBE-SEP-VN2UXCZUKHB4QJN4
-N KUBE-SEP-VWQIYYHOAHHCJU4I
-N KUBE-SEP-YIL6JZP7A3QYXJU2
-N KUBE-SEP-YNKU7LWOWG4XK5JW
-N KUBE-SEP-ZP3FB6NMPNCO4VBJ
-N KUBE-SEP-ZXMNUKOKXUTL2MK2
-N KUBE-SERVICES
-N KUBE-SVC-43N67J7KVPI6PSRJ
-N KUBE-SVC-7WNVDLEMLPUQTTOI
-N KUBE-SVC-E3IBCFULSWKQCT47
-N KUBE-SVC-ERIFXISQEP7F7OF4
-N KUBE-SVC-HX7AG3HOOBNCGV53
-N KUBE-SVC-JD5MR3NA4I4DYORP
-N KUBE-SVC-JLOTPBS4JIX62R47
-N KUBE-SVC-NPX46M4PTMTKRN6Y
-N KUBE-SVC-TCOU7JCQXEZGVUNU
-N KUBE-SVC-ZUD4L6KQKCHD52W4
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
-A KUBE-MARK-DROP -j MARK --set-xmark 0x8000/0x8000
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
-A KUBE-POSTROUTING -m mark ! --mark 0x4000/0x4000 -j RETURN
-A KUBE-POSTROUTING -j MARK --set-xmark 0x4000/0x0
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -j MASQUERADE --random-fully
-A KUBE-SEP-6E7XQMQ4RAYOWTTM -s 10.244.0.3/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
-A KUBE-SEP-6E7XQMQ4RAYOWTTM -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination 10.244.0.3:53
-A KUBE-SEP-HHJOKZF3Y7Y3YORK -s 10.244.1.52/32 -m comment --comment "default/privateregistry:http-web" -j KUBE-MARK-MASQ
-A KUBE-SEP-HHJOKZF3Y7Y3YORK -p tcp -m comment --comment "default/privateregistry:http-web" -m tcp -j DNAT --to-destination 10.244.1.52:5000
-A KUBE-SEP-IT2ZTR26TO4XFPTO -s 10.244.0.2/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
-A KUBE-SEP-IT2ZTR26TO4XFPTO -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination 10.244.0.2:53
-A KUBE-SEP-LV3ODC7YAWYMJH2A -s 10.244.1.19/32 -m comment --comment "default/rabbit:connection" -j KUBE-MARK-MASQ
-A KUBE-SEP-LV3ODC7YAWYMJH2A -p tcp -m comment --comment "default/rabbit:connection" -m tcp -j DNAT --to-destination 10.244.1.19:5672
-A KUBE-SEP-N4G2XR5TDX7PQE7P -s 10.244.0.2/32 -m comment --comment "kube-system/kube-dns:metrics" -j KUBE-MARK-MASQ
-A KUBE-SEP-N4G2XR5TDX7PQE7P -p tcp -m comment --comment "kube-system/kube-dns:metrics" -m tcp -j DNAT --to-destination 10.244.0.2:9153
-A KUBE-SEP-SFKEIAIVHDK3MCWO -s 10.244.1.20/32 -m comment --comment "default/postgres-service:http" -j KUBE-MARK-MASQ
-A KUBE-SEP-SFKEIAIVHDK3MCWO -p tcp -m comment --comment "default/postgres-service:http" -m tcp -j DNAT --to-destination 10.244.1.20:5432
-A KUBE-SEP-V7IHSQL4ET5F356W -s 10.66.66.1/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-V7IHSQL4ET5F356W -p tcp -m comment --comment "default/kubernetes:https" -m tcp -j DNAT --to-destination 10.66.66.1:6443
-A KUBE-SEP-VN2UXCZUKHB4QJN4 -s 10.244.1.19/32 -m comment --comment "default/rabbit:panel" -j KUBE-MARK-MASQ
-A KUBE-SEP-VN2UXCZUKHB4QJN4 -p tcp -m comment --comment "default/rabbit:panel" -m tcp -j DNAT --to-destination 10.244.1.19:15672
-A KUBE-SEP-VWQIYYHOAHHCJU4I -s 10.244.1.49/32 -m comment --comment "cert-manager/cert-manager-webhook:https" -j KUBE-MARK-MASQ
-A KUBE-SEP-VWQIYYHOAHHCJU4I -p tcp -m comment --comment "cert-manager/cert-manager-webhook:https" -m tcp -j DNAT --to-destination 10.244.1.49:10250
-A KUBE-SEP-YIL6JZP7A3QYXJU2 -s 10.244.0.2/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
-A KUBE-SEP-YIL6JZP7A3QYXJU2 -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination 10.244.0.2:53
-A KUBE-SEP-YNKU7LWOWG4XK5JW -s 10.244.1.48/32 -m comment --comment "cert-manager/cert-manager:tcp-prometheus-servicemonitor" -j KUBE-MARK-MASQ
-A KUBE-SEP-YNKU7LWOWG4XK5JW -p tcp -m comment --comment "cert-manager/cert-manager:tcp-prometheus-servicemonitor" -m tcp -j DNAT --to-destination 10.244.1.48:9402
-A KUBE-SEP-ZP3FB6NMPNCO4VBJ -s 10.244.0.3/32 -m comment --comment "kube-system/kube-dns:metrics" -j KUBE-MARK-MASQ
-A KUBE-SEP-ZP3FB6NMPNCO4VBJ -p tcp -m comment --comment "kube-system/kube-dns:metrics" -m tcp -j DNAT --to-destination 10.244.0.3:9153
-A KUBE-SEP-ZXMNUKOKXUTL2MK2 -s 10.244.0.3/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
-A KUBE-SEP-ZXMNUKOKXUTL2MK2 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination 10.244.0.3:53
-A KUBE-SERVICES -d 10.105.208.230/32 -p tcp -m comment --comment "default/postgres-service:http cluster IP" -m tcp --dport 5432 -j KUBE-SVC-HX7AG3HOOBNCGV53
-A KUBE-SERVICES -d 10.102.208.98/32 -p tcp -m comment --comment "cert-manager/cert-manager:tcp-prometheus-servicemonitor cluster IP" -m tcp --dport 9402 -j KUBE-SVC-E3IBCFULSWKQCT47
-A KUBE-SERVICES -d 10.96.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
-A KUBE-SERVICES -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-SVC-ERIFXISQEP7F7OF4
-A KUBE-SERVICES -d 10.103.97.36/32 -p tcp -m comment --comment "default/privateregistry:http-web cluster IP" -m tcp --dport 80 -j KUBE-SVC-JLOTPBS4JIX62R47
-A KUBE-SERVICES -d 10.105.198.60/32 -p tcp -m comment --comment "cert-manager/cert-manager-webhook:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-ZUD4L6KQKCHD52W4
-A KUBE-SERVICES -d 10.100.176.139/32 -p tcp -m comment --comment "default/rabbit:connection cluster IP" -m tcp --dport 5672 -j KUBE-SVC-7WNVDLEMLPUQTTOI
-A KUBE-SERVICES -d 10.100.176.139/32 -p tcp -m comment --comment "default/rabbit:panel cluster IP" -m tcp --dport 15672 -j KUBE-SVC-43N67J7KVPI6PSRJ
-A KUBE-SERVICES -d 10.96.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-TCOU7JCQXEZGVUNU
-A KUBE-SERVICES -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:metrics cluster IP" -m tcp --dport 9153 -j KUBE-SVC-JD5MR3NA4I4DYORP
-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
-A KUBE-SVC-43N67J7KVPI6PSRJ ! -s 10.244.0.0/16 -d 10.100.176.139/32 -p tcp -m comment --comment "default/rabbit:panel cluster IP" -m tcp --dport 15672 -j KUBE-MARK-MASQ
-A KUBE-SVC-43N67J7KVPI6PSRJ -m comment --comment "default/rabbit:panel -> 10.244.1.19:15672" -j KUBE-SEP-VN2UXCZUKHB4QJN4
-A KUBE-SVC-7WNVDLEMLPUQTTOI ! -s 10.244.0.0/16 -d 10.100.176.139/32 -p tcp -m comment --comment "default/rabbit:connection cluster IP" -m tcp --dport 5672 -j KUBE-MARK-MASQ
-A KUBE-SVC-7WNVDLEMLPUQTTOI -m comment --comment "default/rabbit:connection -> 10.244.1.19:5672" -j KUBE-SEP-LV3ODC7YAWYMJH2A
-A KUBE-SVC-E3IBCFULSWKQCT47 ! -s 10.244.0.0/16 -d 10.102.208.98/32 -p tcp -m comment --comment "cert-manager/cert-manager:tcp-prometheus-servicemonitor cluster IP" -m tcp --dport 9402 -j KUBE-MARK-MASQ
-A KUBE-SVC-E3IBCFULSWKQCT47 -m comment --comment "cert-manager/cert-manager:tcp-prometheus-servicemonitor -> 10.244.1.48:9402" -j KUBE-SEP-YNKU7LWOWG4XK5JW
-A KUBE-SVC-ERIFXISQEP7F7OF4 ! -s 10.244.0.0/16 -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp -> 10.244.0.2:53" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-IT2ZTR26TO4XFPTO
-A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp -> 10.244.0.3:53" -j KUBE-SEP-ZXMNUKOKXUTL2MK2
-A KUBE-SVC-HX7AG3HOOBNCGV53 ! -s 10.244.0.0/16 -d 10.105.208.230/32 -p tcp -m comment --comment "default/postgres-service:http cluster IP" -m tcp --dport 5432 -j KUBE-MARK-MASQ
-A KUBE-SVC-HX7AG3HOOBNCGV53 -m comment --comment "default/postgres-service:http -> 10.244.1.20:5432" -j KUBE-SEP-SFKEIAIVHDK3MCWO
-A KUBE-SVC-JD5MR3NA4I4DYORP ! -s 10.244.0.0/16 -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:metrics cluster IP" -m tcp --dport 9153 -j KUBE-MARK-MASQ
-A KUBE-SVC-JD5MR3NA4I4DYORP -m comment --comment "kube-system/kube-dns:metrics -> 10.244.0.2:9153" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-N4G2XR5TDX7PQE7P
-A KUBE-SVC-JD5MR3NA4I4DYORP -m comment --comment "kube-system/kube-dns:metrics -> 10.244.0.3:9153" -j KUBE-SEP-ZP3FB6NMPNCO4VBJ
-A KUBE-SVC-JLOTPBS4JIX62R47 ! -s 10.244.0.0/16 -d 10.103.97.36/32 -p tcp -m comment --comment "default/privateregistry:http-web cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
-A KUBE-SVC-JLOTPBS4JIX62R47 -m comment --comment "default/privateregistry:http-web -> 10.244.1.52:5000" -j KUBE-SEP-HHJOKZF3Y7Y3YORK
-A KUBE-SVC-NPX46M4PTMTKRN6Y ! -s 10.244.0.0/16 -d 10.96.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https -> 10.66.66.1:6443" -j KUBE-SEP-V7IHSQL4ET5F356W
-A KUBE-SVC-TCOU7JCQXEZGVUNU ! -s 10.244.0.0/16 -d 10.96.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-MARK-MASQ
-A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns -> 10.244.0.2:53" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-YIL6JZP7A3QYXJU2
-A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns -> 10.244.0.3:53" -j KUBE-SEP-6E7XQMQ4RAYOWTTM
-A KUBE-SVC-ZUD4L6KQKCHD52W4 ! -s 10.244.0.0/16 -d 10.105.198.60/32 -p tcp -m comment --comment "cert-manager/cert-manager-webhook:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
-A KUBE-SVC-ZUD4L6KQKCHD52W4 -m comment --comment "cert-manager/cert-manager-webhook:https -> 10.244.1.49:10250" -j KUBE-SEP-VWQIYYHOAHHCJU4I
Could it be an MTU problem? I am trying to understand your setup but it could be that the packets forwarded through the tunnel could be dropped if they exceed the MTU size.
Hello, thank you for your answer,
My nodes that are running without any problem have a mtu of 1420 on wg0 (wireguard interface). However the node that is not running correctly have a mtu of 1320.
Could it be the reason?
Are you using vxlan as backend on flannel? The MTU is only a possible reason for your issue because it fails on TLS handshake and the TCP handshake works fine. My hypothesis for the MTU is related to the fact that the HELLO packet for the TLS handshake is bigger then the TCP handshake packets so it could be dropped for exceeding the MTU. You can try to capture the traffic and follow the TCP stream to the apiserver and also check the drop counter of the net interfaces.
I think I am using vxlan since I followed the basic installation.
I tried to capture the traffic using tcpdump and here is what im getting:
13:37:08.743590 IP 10.66.66.1.44226 > 10.66.66.3.10250: Flags [.], ack 3317, win 502, options [nop,nop,TS val 2250034702 ecr 2517146376], length 0
13:37:08.745803 IP 10.66.66.1.44226 > 10.66.66.3.10250: Flags [.], ack 3962, win 497, options [nop,nop,TS val 2250034704 ecr 2517146377], length 0
13:37:08.745804 IP 10.66.66.1.44226 > 10.66.66.3.10250: Flags [.], ack 4168, win 497, options [nop,nop,TS val 2250034704 ecr 2517146377], length 0
13:37:09.705381 IP 10.66.66.1.44226 > 10.66.66.3.10250: Flags [P.], seq 1803:1983, ack 4168, win 502, options [nop,nop,TS val 2250035663 ecr 2517146377], length 180
13:37:09.705417 IP 10.66.66.3.10250 > 10.66.66.1.44226: Flags [.], ack 1983, win 501, options [nop,nop,TS val 2517147355 ecr 2250035663], length 0
13:37:09.705978 IP 10.66.66.3.10250 > 10.66.66.1.44226: Flags [P.], seq 4168:4274, ack 1983, win 502, options [nop,nop,TS val 2517147356 ecr 2250035663], length 106
13:37:09.707196 IP 10.66.66.3.10250 > 10.66.66.1.44226: Flags [P.], seq 4274:4919, ack 1983, win 502, options [nop,nop,TS val 2517147357 ecr 2250035663], length 645
13:37:09.707314 IP 10.66.66.3.10250 > 10.66.66.1.44226: Flags [P.], seq 4919:5125, ack 1983, win 502, options [nop,nop,TS val 2517147357 ecr 2250035663], length 206
13:37:09.728161 IP 10.66.66.1.44226 > 10.66.66.3.10250: Flags [.], ack 4274, win 502, options [nop,nop,TS val 2250035686 ecr 2517147356], length 0
13:37:09.728179 IP 10.66.66.1.44226 > 10.66.66.3.10250: Flags [.], ack 4919, win 497, options [nop,nop,TS val 2250035686 ecr 2517147357], length 0
13:37:09.728179 IP 10.66.66.1.44226 > 10.66.66.3.10250: Flags [.], ack 5125, win 496, options [nop,nop,TS val 2250035686 ecr 2517147357], length 0
13:37:12.305187 IP 10.66.66.3.59616 > 10.66.66.1.6443: Flags [P.], seq 7753:7791, ack 89115, win 3305, options [nop,nop,TS val 2517149955 ecr 2250033474], length 38
13:37:12.306686 IP 10.66.66.3.59616 > 10.66.66.1.6443: Flags [P.], seq 7791:7831, ack 89115, win 3305, options [nop,nop,TS val 2517149956 ecr 2250033474], length 40
13:37:12.306742 IP 10.66.66.3.59616 > 10.66.66.1.6443: Flags [P.], seq 7831:8360, ack 89115, win 3305, options [nop,nop,TS val 2517149956 ecr 2250033474], length 529
13:37:12.326634 IP 10.66.66.1.6443 > 10.66.66.3.59616: Flags [.], ack 8360, win 502, options [nop,nop,TS val 2250038285 ecr 2517149955], length 0
13:37:12.331449 IP 10.66.66.1.6443 > 10.66.66.3.59616: Flags [P.], seq 89115:89205, ack 8360, win 502, options [nop,nop,TS val 2250038289 ecr 2517149955], length 90
13:37:12.331487 IP 10.66.66.3.59616 > 10.66.66.1.6443: Flags [.], ack 89205, win 3305, options [nop,nop,TS val 2517149981 ecr 2250038289], length 0
13:37:12.331793 IP 10.66.66.1.6443 > 10.66.66.3.59616: Flags [.], seq 89205:90473, ack 8360, win 502, options [nop,nop,TS val 2250038290 ecr 2517149955], length 1268
13:37:12.331814 IP 10.66.66.3.59616 > 10.66.66.1.6443: Flags [.], ack 90473, win 3300, options [nop,nop,TS val 2517149982 ecr 2250038290], length 0
13:37:12.331855 IP 10.66.66.1.6443 > 10.66.66.3.59616: Flags [P.], seq 90473:91741, ack 8360, win 502, options [nop,nop,TS val 2250038290 ecr 2517149955], length 1268
13:37:12.331870 IP 10.66.66.3.59616 > 10.66.66.1.6443: Flags [.], ack 91741, win 3291, options [nop,nop,TS val 2517149982 ecr 2250038290], length 0
13:37:12.331941 IP 10.66.66.1.6443 > 10.66.66.3.59616: Flags [P.], seq 91741:92332, ack 8360, win 502, options [nop,nop,TS val 2250038290 ecr 2517149955], length 591
13:37:12.331958 IP 10.66.66.3.59616 > 10.66.66.1.6443: Flags [.], ack 92332, win 3300, options [nop,nop,TS val 2517149982 ecr 2250038290], length 0
13:37:12.334008 IP 10.66.66.1.6443 > 10.66.66.3.59616: Flags [P.], seq 92332:92367, ack 8360, win 502, options [nop,nop,TS val 2250038292 ecr 2517149955], length 35
13:37:12.334037 IP 10.66.66.3.59616 > 10.66.66.1.6443: Flags [.], ack 92367, win 3304, options [nop,nop,TS val 2517149984 ecr 2250038292], length 0
13:37:12.340175 IP 10.66.66.1.6443 > 10.66.66.3.59616: Flags [P.], seq 92367:92434, ack 8360, win 502, options [nop,nop,TS val 2250038298 ecr 2517149955], length 67
13:37:12.340189 IP 10.66.66.3.59616 > 10.66.66.1.6443: Flags [.], ack 92434, win 3304, options [nop,nop,TS val 2517149990 ecr 2250038298], length 0
13:37:12.340230 IP 10.66.66.1.6443 > 10.66.66.3.59616: Flags [P.], seq 92434:92963, ack 8360, win 502, options [nop,nop,TS val 2250038299 ecr 2517149955], length 529
13:37:12.340239 IP 10.66.66.3.59616 > 10.66.66.1.6443: Flags [.], ack 92963, win 3300, options [nop,nop,TS val 2517149990 ecr 2250038299], length 0
13:37:12.948390 IP 10.66.66.1.6443 > 10.66.66.3.49477: Flags [.], seq 1904:3252, ack 1460, win 502, options [nop,nop,TS val 2250038183 ecr 2183768355], length 1348
13:37:12.948465 IP 10.66.66.3.49477 > 10.66.66.1.6443: Flags [.], ack 3252, win 492, options [nop,nop,TS val 2183786231 ecr 2250038183,nop,nop,sack 2 {5948:8018}{12062:12537}], length 0
13:37:15.461730 IP 10.66.66.3.33126 > 10.66.66.1.6443: Flags [P.], seq 39:78, ack 40, win 562, options [nop,nop,TS val 2517153111 ecr 2250011419], length 39
13:37:15.483275 IP 10.66.66.1.6443 > 10.66.66.3.33126: Flags [P.], seq 40:79, ack 78, win 502, options [nop,nop,TS val 2250041441 ecr 2517153111], length 39
13:37:15.483311 IP 10.66.66.3.33126 > 10.66.66.1.6443: Flags [.], ack 79, win 562, options [nop,nop,TS val 2517153133 ecr 2250041441], length 0
^C
353 packets captured
353 packets received by filter
0 packets dropped by kerne
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}