edgemesh icon indicating copy to clipboard operation
edgemesh copied to clipboard
verified publisher icon kubeedge

websocket connection failing

Open vikarna opened this issue 3 years ago • 6 comments

What happened: Websocket connection fails over edgemesh-gw and vs What you expected to happen: websocket connection should get established How to reproduce it (as minimally and precisely as possible): configure gw,vs,svc and pods for websocket usage Anything else we need to know?:

########edgemesh gateway config############ apiVersion: networking.istio.io/v1beta1 kind: Gateway metadata: name: edgemesh-gateway namespace: default spec: selector: kubeedge: edgemesh-gateway servers:

  • hosts:
    • '*' port: name: http-0 number: 80 protocol: HTTP

#########virtual service############## apiVersion: networking.istio.io/v1beta1 kind: VirtualService metadata: name: edgemesh-gateway-svc namespace: default spec: gateways:

  • edgemesh-gateway hosts:
  • '*' http:
  • match:
    • uri: prefix: /echo route:
    • destination: host: ws-svc port: number: 12348

##########service############# apiVersion: v1 kind: Service metadata: name: ws-svc namespace: default spec: clusterIP: 10.97.61.44 clusterIPs:

  • 10.97.61.44 ipFamilies:
  • IPv4 ipFamilyPolicy: SingleStack ports:
  • name: http-0 port: 12348 protocol: TCP targetPort: 8080 selector: app: ws-edge sessionAffinity: None type: ClusterIP status: loadBalancer: {}

#########################Deployment###### Deployment is from the edgemesh test cases. The test case runs successsfully inside the cluster but as soon as i try to access it from outside the cluster by using gw and vs, it fails.

############output from docker container outside the cluster######## / # ./client --addr 192.168.64.18 connecting to ws://192.168.64.18/echo read: websocket: close 1006 (abnormal closure): unexpected EOF / #

vikarna avatar May 07 '22 19:05 vikarna

Please provide edgemesh-agent logs.

Poorunga avatar May 10 '22 02:05 Poorunga

#######edgemesh-gw######### I0509 23:27:24.846112 1 http.go:137] destination svc is default.ws-svc I0509 23:27:24.846250 1 log.go:184] DEBUG: add [2] handlers for chain [tcp] I0509 23:27:24.846343 1 handler.go:58] l4 proxy get tcpserver address: worker:10.85.0.96:8080 I0509 23:27:24.866422 1 handler.go:108] l4 proxy start proxy data between tcpserver worker:10.85.0.96:8080 I0509 23:27:24.866470 1 handler.go:120] Success proxy to worker:10.85.0.96:8080 I0509 23:28:38.352809 1 http.go:137] destination svc is default.ws-svc I0509 23:28:38.352957 1 log.go:184] DEBUG: add [2] handlers for chain [tcp] I0509 23:28:38.352991 1 handler.go:58] l4 proxy get tcpserver address: worker:10.85.0.96:8080 I0509 23:28:38.354512 1 handler.go:108] l4 proxy start proxy data between tcpserver worker:10.85.0.96:8080 I0509 23:28:38.354529 1 handler.go:120] Success proxy to worker:10.85.0.96:8080

################edgmesh agent#################### I0509 23:27:24.826884 1 proxy.go:47] Got a new stream from QmQK5sWzFhcoPkD2E8CL7tofmwgYpHkRxRhbj8iPpUfKri I0509 23:27:24.828548 1 proxy.go:92] Success proxy for {tcp worker 10.85.0.96:8080} I0509 23:28:38.317730 1 proxy.go:47] Got a new stream from QmQK5sWzFhcoPkD2E8CL7tofmwgYpHkRxRhbj8iPpUfKri I0509 23:28:38.318045 1 proxy.go:92] Success proxy for {tcp worker 10.85.0.96:8080}

##############Pod####### ws-edge-5978d75769-ncxkf 1/1 Running 0 2d 10.85.0.96 worker

vikarna avatar May 10 '22 04:05 vikarna

Need full edgemesh-agent log, when you use ./client --addr 192.168.64.18 command.

BTW, you are missing the port in ./client --addr 192.168.64.18.

Poorunga avatar May 10 '22 07:05 Poorunga

I tried both with and without port. / # ./client --addr 192.168.64.18 connecting to ws://192.168.64.18/echo read: websocket: close 1006 (abnormal closure): unexpected EOF / # ./client --addr 192.168.64.18:80 connecting to ws://192.168.64.18:80/echo read: websocket: close 1006 (abnormal closure): unexpected EOF / #

Not much logs are seen. Everytime the last two lines are getting added. #####Logs####### I0509 23:26:04.240551 1 tunnel.go:31] Tunnel agent connecting to tunnel server I0509 23:26:04.240601 1 module.go:65] Runs CoreDNS v1.8.7 as a local dns I0509 23:26:04.242054 1 config.go:307] "Starting service config controller" I0509 23:26:04.242116 1 shared_informer.go:240] Waiting for caches to sync for service config I0509 23:26:04.242361 1 config.go:125] "Starting endpoints config controller" I0509 23:26:04.242370 1 shared_informer.go:240] Waiting for caches to sync for endpoints config I0509 23:26:04.242380 1 config.go:544] "Starting destination rule config controller" I0509 23:26:04.242385 1 shared_informer.go:240] Waiting for caches to sync for destination rule config I0509 23:26:04.341007 1 tunnel.go:50] agent success connected to server [/ip4/192.168.64.18/tcp/20004 /ip4/127.0.0.1/tcp/20004 /ip4/103.208.69.161/tcp/20004] I0509 23:26:04.342235 1 shared_informer.go:247] Caches are synced for service config I0509 23:26:04.342674 1 shared_informer.go:247] Caches are synced for destination rule config I0509 23:26:04.342723 1 shared_informer.go:247] Caches are synced for endpoints config .:53 on 169.254.96.16 [INFO] plugin/reload: Running configuration MD5 = 0b3d85a9c43336189e8532480396fd1f CoreDNS-1.8.7 linux/arm64, go1.17.9, [INFO] 169.254.96.16:52524 - 56860 "HINFO IN 1800517704851445788.6511668148845874275. udp 57 false 512" NXDOMAIN qr,rd,ra 57 0.09014604s I0509 23:26:04.932998 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="ingress-nginx/ingress-nginx-controller:http" protocol=TCP nodePort=30854 I0509 23:26:04.935532 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="ingress-nginx/ingress-nginx-controller:http" protocol=TCP nodePort=30854 I0509 23:26:04.938406 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="ingress-nginx/ingress-nginx-controller:http" protocol=TCP nodePort=30854 I0509 23:26:04.944043 1 proxier.go:898] "Opened iptables from-containers public port for service" servicePortName="ingress-nginx/ingress-nginx-controller:https" protocol=TCP nodePort=32525 I0509 23:26:04.945966 1 proxier.go:909] "Opened iptables from-host public port for service" servicePortName="ingress-nginx/ingress-nginx-controller:https" protocol=TCP nodePort=32525 I0509 23:26:04.947992 1 proxier.go:919] "Opened iptables from-non-local public port for service" servicePortName="ingress-nginx/ingress-nginx-controller:https" protocol=TCP nodePort=32525 I0509 23:27:24.826884 1 proxy.go:47] Got a new stream from QmQK5sWzFhcoPkD2E8CL7tofmwgYpHkRxRhbj8iPpUfKri I0509 23:27:24.828548 1 proxy.go:92] Success proxy for {tcp worker 10.85.0.96:8080} I0509 23:28:38.317730 1 proxy.go:47] Got a new stream from QmQK5sWzFhcoPkD2E8CL7tofmwgYpHkRxRhbj8iPpUfKri I0509 23:28:38.318045 1 proxy.go:92] Success proxy for {tcp worker 10.85.0.96:8080} I0510 13:18:01.218843 1 proxy.go:47] Got a new stream from QmQK5sWzFhcoPkD2E8CL7tofmwgYpHkRxRhbj8iPpUfKri I0510 13:18:01.221746 1 proxy.go:92] Success proxy for {tcp worker 10.85.0.96:8080} I0510 13:18:37.160474 1 proxy.go:47] Got a new stream from QmQK5sWzFhcoPkD2E8CL7tofmwgYpHkRxRhbj8iPpUfKri I0510 13:18:37.161362 1 proxy.go:92] Success proxy for {tcp worker 10.85.0.96:8080}

vikarna avatar May 10 '22 07:05 vikarna

192.168.64.18 is your websocket service's clusterip?

Poorunga avatar May 10 '22 09:05 Poorunga

No, It's the node IP on which edgemesh gw is running. I have given the gw config above.

vikarna avatar May 10 '22 12:05 vikarna