edgemesh icon indicating copy to clipboard operation
edgemesh copied to clipboard
verified publisher icon kubeedge

Connect to server err: failed to dial(127.0.0.1:20004: connect: connection refused)

Open flyme001 opened this issue 3 years ago • 3 comments

k8s v1.21 kubeedge 1.9.2 edgemesh latest

[root@vm10 ~]# kubectl logs -f edgemesh-agent-4s9w7 -n kubeedge

2022/08/04 02:19:32 INFO: Install client plugin, protocol: rest 2022/08/04 02:19:32 INFO: Installed service discovery plugin: edge I0804 02:19:32.427675 1 server.go:55] Version: v1.11.0-dirty I0804 02:19:32.427704 1 server.go:89] [1] New informers manager I0804 02:19:32.428300 1 server.go:96] [2] Prepare agent to run I0804 02:19:32.428582 1 netif.go:98] bridge device edgemesh0 already exists I0804 02:19:32.428675 1 server.go:100] edgemesh-agent running on EdgeMode I0804 02:19:32.428694 1 server.go:110] [3] Register beehive modules I0804 02:19:32.429397 1 module.go:34] Module edgedns registered successfully I0804 02:19:32.437770 1 server.go:87] Using userspace Proxier. I0804 02:19:32.456680 1 module.go:34] Module edgeproxy registered successfully W0804 02:19:32.456693 1 module.go:37] Module edgegateway is disabled, do not register I0804 02:19:32.456771 1 manager.go:33] Use noCaManager to handle security I0804 02:19:32.457210 1 nocamanager.go:44] Private key exist, skip generate I0804 02:19:32.461514 1 module.go:86] Listening on /ip4/192.168.0.42/tcp/20006/p2p/QmXhvKxeAD7NqnEyNrPtwdpLDb1aT2yGGe2LJANHcjuG94 I0804 02:19:32.461537 1 module.go:86] Listening on /ip4/127.0.0.1/tcp/20006/p2p/QmXhvKxeAD7NqnEyNrPtwdpLDb1aT2yGGe2LJANHcjuG94 I0804 02:19:32.461549 1 module.go:34] Module tunnelagent registered successfully I0804 02:19:32.461554 1 server.go:116] [4] Start informers manager I0804 02:19:32.762559 1 server.go:120] [5] Start all modules I0804 02:19:32.762747 1 core.go:24] Starting module tunnelagent I0804 02:19:32.762885 1 core.go:24] Starting module edgedns I0804 02:19:32.763134 1 core.go:24] Starting module edgeproxy I0804 02:19:32.763913 1 module.go:65] Runs CoreDNS v1.8.7 as a local dns W0804 02:19:32.766738 1 tunnel.go:28] Connection between agent and server [/ip4/192.168.0.30/tcp/20004 /ip4/127.0.0.1/tcp/20004] is not established, try connect I0804 02:19:32.766767 1 tunnel.go:31] Tunnel agent connecting to tunnel server I0804 02:19:32.769042 1 config.go:307] "Starting service config controller" I0804 02:19:32.769058 1 shared_informer.go:240] Waiting for caches to sync for service config W0804 02:19:32.769156 1 tunnel.go:34] Connect to server err: failed to dial QmasLHSpgLzXmZayCTxyzK6GdabYXt169MvJypa11Dg5tr: all dials failed

  • [/ip4/127.0.0.1/tcp/20004] dial tcp4 127.0.0.1:20004: connect: connection refused
  • [/ip4/192.168.0.30/tcp/20004] dial tcp4 192.168.0.30:20004: connect: connection refused I0804 02:19:32.769536 1 config.go:125] "Starting endpoints config controller" I0804 02:19:32.769548 1 shared_informer.go:240] Waiting for caches to sync for endpoints config I0804 02:19:32.769563 1 config.go:544] "Starting destination rule config controller" I0804 02:19:32.769568 1 shared_informer.go:240] Waiting for caches to sync for destination rule config .:53 on 169.254.96.16 [INFO] plugin/reload: Running configuration MD5 = 0b3d85a9c43336189e8532480396fd1f CoreDNS-1.8.7 linux/amd64, go1.17.11, I0804 02:19:32.870425 1 shared_informer.go:247] Caches are synced for destination rule config I0804 02:19:32.870457 1 shared_informer.go:247] Caches are synced for service config I0804 02:19:32.870476 1 shared_informer.go:247] Caches are synced for endpoints config I0804 02:19:32.886916 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/emqx:mqtt" protocol=TCP nodePort=31883 I0804 02:19:32.892528 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/emqx:mqtt" protocol=TCP nodePort=31883 I0804 02:19:32.896826 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/emqx:mqtt" protocol=TCP nodePort=31883 I0804 02:19:32.909136 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/emqx:unknown-1" protocol=TCP nodePort=31377 I0804 02:19:32.913229 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/emqx:unknown-1" protocol=TCP nodePort=31377 I0804 02:19:32.917012 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/emqx:unknown-1" protocol=TCP nodePort=31377 I0804 02:19:32.925907 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/emqx:unknown-2" protocol=TCP nodePort=31178 I0804 02:19:32.929193 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/emqx:unknown-2" protocol=TCP nodePort=31178 I0804 02:19:32.933521 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/emqx:unknown-2" protocol=TCP nodePort=31178 I0804 02:19:32.943865 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/emqx:unknown-3" protocol=TCP nodePort=31145 I0804 02:19:32.946867 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/emqx:unknown-3" protocol=TCP nodePort=31145 I0804 02:19:32.950065 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/emqx:unknown-3" protocol=TCP nodePort=31145 I0804 02:19:32.966908 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/emqx:unknown-4" protocol=TCP nodePort=30508 I0804 02:19:32.969716 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/emqx:unknown-4" protocol=TCP nodePort=30508 I0804 02:19:32.972741 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/emqx:unknown-4" protocol=TCP nodePort=30508 I0804 02:19:32.986466 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/prometheus:http" protocol=TCP nodePort=30900 I0804 02:19:32.989616 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/prometheus:http" protocol=TCP nodePort=30900 I0804 02:19:32.992485 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/prometheus:http" protocol=TCP nodePort=30900 I0804 02:19:33.000261 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/redis:redis" protocol=TCP nodePort=30566 I0804 02:19:33.002573 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/redis:redis" protocol=TCP nodePort=30566 I0804 02:19:33.005261 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/redis:redis" protocol=TCP nodePort=30566 I0804 02:19:33.017420 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/my-kuiper:restapi" protocol=TCP nodePort=31453 I0804 02:19:33.019887 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/my-kuiper:restapi" protocol=TCP nodePort=31453 I0804 02:19:33.023161 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/my-kuiper:restapi" protocol=TCP nodePort=31453 I0804 02:19:33.035238 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/my-kuiper:kuiper" protocol=TCP nodePort=30689 I0804 02:19:33.037438 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/my-kuiper:kuiper" protocol=TCP nodePort=30689 I0804 02:19:33.039937 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/my-kuiper:kuiper" protocol=TCP nodePort=30689 I0804 02:19:33.062879 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/mysql" protocol=TCP nodePort=32162 I0804 02:19:33.066825 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/mysql" protocol=TCP nodePort=32162 I0804 02:19:33.070529 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/mysql" protocol=TCP nodePort=32162 [INFO] 169.254.96.16:50186 - 47545 "HINFO IN 8933918750094615264.2180968571933027549. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.247721986s I0804 02:19:33.127641 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/webserver" protocol=TCP nodePort=31270 I0804 02:19:33.131081 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/webserver" protocol=TCP nodePort=31270 I0804 02:19:33.134542 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/webserver" protocol=TCP nodePort=31270 I0804 02:19:33.144568 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="default/grafana:http" protocol=TCP nodePort=30300 I0804 02:19:33.148300 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="default/grafana:http" protocol=TCP nodePort=30300 I0804 02:19:33.151152 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="default/grafana:http" protocol=TCP nodePort=30300 I0804 02:19:34.769793 1 tunnel.go:31] Tunnel agent connecting to tunnel server W0804 02:19:34.770092 1 tunnel.go:34] Connect to server err: dial backoff I0804 02:19:36.770957 1 tunnel.go:31] Tunnel agent connecting to tunnel server W0804 02:19:36.771183 1 tunnel.go:34] Connect to server err: dial backoff W0804 02:19:48.772041 1 tunnel.go:28] Connection between agent and server [/ip4/192.168.0.30/tcp/20004 /ip4/127.0.0.1/tcp/20004] is not established, try connect I0804 02:19:48.772122 1 tunnel.go:31] Tunnel agent connecting to tunnel server W0804 02:19:48.773350 1 tunnel.go:34] Connect to server err: failed to dial QmasLHSpgLzXmZayCTxyzK6GdabYXt169MvJypa11Dg5tr: all dials failed
  • [/ip4/127.0.0.1/tcp/20004] dial tcp4 127.0.0.1:20004: connect: connection refused
  • [/ip4/192.168.0.30/tcp/20004] dial tcp4 192.168.0.30:20004: connect: connection refused I0804 02:19:50.774290 1 tunnel.go:31] Tunnel agent connecting to tunnel server W0804 02:19:50.774449 1 tunnel.go:34] Connect to server err: dial backoff I0804 02:19:52.775201 1 tunnel.go:31] Tunnel agent connecting to tunnel server W0804 02:19:52.775451 1 tunnel.go:34] Connect to server err: dial backoff W0804 02:20:04.777074 1 tunnel.go:28] Connection between agent and server [/ip4/192.168.0.30/tcp/20004 /ip4/127.0.0.1/tcp/20004] is not established, try connect I0804 02:20:04.777112 1 tunnel.go:31] Tunnel agent connecting to tunnel server I0804 02:20:04.784516 1 tunnel.go:50] agent success connected to server [/ip4/192.168.0.30/tcp/20004 /ip4/127.0.0.1/tcp/20004]

flyme001 avatar Aug 04 '22 07:08 flyme001

edgemesh demo 走了下 在edge上部署http 服务 [root@vm10 ~]# kubectl get pod hostname-edge-84cb45ccf4-sxz54 -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES hostname-edge-84cb45ccf4-sxz54 1/1 Running 0 57m 172.17.0.6 vm-42 [root@vm10 ~]# kubectl get svc hostname-svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE hostname-svc ClusterIP 10.233.25.49 12345/TCP 57m

在cloud 端curl 不通 [root@vm10 ~]# kubectl get pods webserver-798b5bf886-95mvv -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES webserver-798b5bf886-95mvv 1/1 Running 11 35d 10.233.93.174 vm10 [root@vm10 ~]# kubectl exec -it webserver-798b5bf886-95mvv sh kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. / # curl hostname-svc:12345

/ # nslookup hostname-svc.default.svc.cluster.local Server: 169.254.25.10 Address: 169.254.25.10:53

Name: hostname-svc.default.svc.cluster.local Address: 10.233.25.49

flyme001 avatar Aug 04 '22 08:08 flyme001

看了下网上解决方案 rabc cloudcore 增加namespace 但是keadm 安装没有这个rabc

flyme001 avatar Aug 04 '22 08:08 flyme001

edgemesh-server的advertiseAddress参数填写了吗？

Poorunga avatar Aug 04 '22 15:08 Poorunga