crane icon indicating copy to clipboard operation
crane copied to clipboard

fix nginx start fail when upstream does not work

Open zsnmwy opened this issue 2 years ago • 2 comments

What type of PR is this?

fix Nginx start fail when upstream does not work

What this PR does / why we need it:

  1. Use Lua to get the nameserver address and cluster domain in /etc/resolv.conf when Nginx is in init_by_lua.
  2. Query domain in access_by_lua.And cache the DNS answers in the 300s.
  3. Set upstream in balancer_by_lua

image

# Get env from resolv.conf
2022/06/01 14:52:51 [notice] 1#1: [lua] get_nameserver.lua:42: _read_dns_servers_from_resolv_file(): found dns server:10.43.0.10
2022/06/01 14:52:51 [notice] 1#1: [lua] get_nameserver.lua:49: _read_dns_servers_from_resolv_file(): cluster_domain:crane-system.svc.cluster.local
2022/06/01 14:52:51 [notice] 1#1: using the "epoll" event method
2022/06/01 14:52:51 [notice] 1#1: openresty/1.21.4.1
2022/06/01 14:52:51 [notice] 1#1: built by gcc 10.3.1 20211027 (Alpine 10.3.1_git20211027)
2022/06/01 14:52:51 [notice] 1#1: OS: Linux 4.15.0-159-generic
2022/06/01 14:52:51 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2022/06/01 14:52:51 [notice] 1#1: start worker processes
2022/06/01 14:52:51 [notice] 1#1: start worker process 8

# grafana not install
2022/06/01 14:53:39 [notice] 8#8: *1 "/grafana/(.*)" does not match "/grafana", client: 127.0.0.1, server: _, request: "GET /grafana HTTP/1.1", host: "127.0.0.1:9090"
2022/06/01 14:53:39 [error] 8#8: *1 [lua] dns.lua:38: get_addr(): Failed to query hostname: grafana.crane-system.svc.cluster.local errcode: 3, client: 127.0.0.1, server: _, request: "GET /g
2022/06/01 14:53:39 [crit] 8#8: *1 connect() to 0.0.0.1:80 failed (22: Invalid argument) while connecting to upstream, client: 127.0.0.1, server: _, request: "GET /grafana HTTP/1.1", upstre
127.0.0.1 - - [01/Jun/2022:14:53:39 +0000] "GET /grafana HTTP/1.1" 502 163 "-" "curl/7.80.0"
2022/06/01 14:53:39 [info] 8#8: *1 client 127.0.0.1 closed keepalive connection

# install grafana - first access
2022/06/01 14:55:41 [notice] 8#8: *4 "/grafana/(.*)" does not match "/grafana", client: 127.0.0.1, server: _, request: "GET /grafana HTTP/1.1", host: "127.0.0.1:9090"
2022/06/01 14:55:41 [notice] 8#8: *4 [lua] dns.lua:45: get_addr(): Query DNS: grafana => 10.43.66.91, client: 127.0.0.1, server: _, request: "GET /grafana HTTP/1.1", host: "127.0.0.1:9090"
2022/06/01 14:55:41 [notice] 8#8: *4 [lua] lb.lua:13: set_domain(): set domain: grafana set IP: 10.43.66.91 Port: 8082 while connecting to upstream, client: 127.0.0.1, server: _, request: "
127.0.0.1 - - [01/Jun/2022:14:55:41 +0000] "GET /grafana HTTP/1.1" 200 31581 "-" "curl/7.80.0"
2022/06/01 14:55:41 [info] 8#8: *4 client 127.0.0.1 closed keepalive connection

# install granfana - second access
2022/06/01 14:55:43 [notice] 8#8: *7 "/grafana/(.*)" does not match "/grafana", client: 127.0.0.1, server: _, request: "GET /grafana HTTP/1.1", host: "127.0.0.1:9090"
2022/06/01 14:55:43 [notice] 8#8: *7 [lua] lb.lua:13: set_domain(): set domain: grafana set IP: 10.43.66.91 Port: 8082 while connecting to upstream, client: 127.0.0.1, server: _, request: "
127.0.0.1 - - [01/Jun/2022:14:55:43 +0000] "GET /grafana HTTP/1.1" 200 31581 "-" "curl/7.80.0"
2022/06/01 14:55:43 [info] 8#8: *7 client 127.0.0.1 closed keepalive connection

Which issue(s) this PR fixes:

Fixes #235

Special notes for your reviewer:

zsnmwy avatar Jun 01 '22 15:06 zsnmwy

🎉 Successfully Build Images.

Overview: https://finops.coding.net/public-artifacts/gocrane/crane/packages

Image Pull Command
crane-agent:pr-343-0d82707 docker pull finops-docker.pkg.coding.net/gocrane/crane/crane-agent:pr-343-0d82707
dashboard:pr-343-0d82707 docker pull finops-docker.pkg.coding.net/gocrane/crane/dashboard:pr-343-0d82707
metric-adapter:pr-343-0d82707 docker pull finops-docker.pkg.coding.net/gocrane/crane/metric-adapter:pr-343-0d82707
craned:pr-343-0d82707 docker pull finops-docker.pkg.coding.net/gocrane/crane/craned:pr-343-0d82707

github-actions[bot] avatar Jun 01 '22 15:06 github-actions[bot]

Missing update ConfigMap

lbbniu avatar Jun 04 '22 05:06 lbbniu