istio icon indicating copy to clipboard operation
istio copied to clipboard

Upgrade istio from 1.8.1 to 1.10.1 with Helm

Open liyuntian opened this issue 3 years ago • 2 comments

Bug Description

first I install ingressgateway in foo8 namespace helm install istio-ingress manifests/charts/gateways/istio-ingress
--set global.hub="xxxxxx/istio8"
--set global.tag="1.8.1"
--set global.imagePullPolicy="IfNotPresent"
--set gateways.istio-ingressgateway.autoscaleMin=2
--set global.jwtPolicy=first-party-jwt
-n foo8

then I install canary istio helm install istiod-canary manifests/charts/istio-control/istio-discovery
--set global.hub="xxx/istio10"
--set global.tag="1.10.1"
--set global.imagePullPolicy="IfNotPresent"
--set pilot.autoscaleMin=2
--set global.jwtPolicy=first-party-jwt
--set revision=canary
--set pilot.env.PILOT_SCOPE_GATEWAY_TO_NAMESPACE=true
-n istio-system

now istiod and istio-canary can run well

I label namespace foo10 istio.io/rev=canary But,Now I want to install ingressgateway in foo10 helm install istio-ingress manifests/charts/gateways/istio-ingress
--set global.hub="xxxx/istio10"
--set global.tag="1.10.1"
--set revision="canary"
--set global.imagePullPolicy="IfNotPresent"
--set gateways.istio-ingressgateway.autoscaleMin=2
--set global.jwtPolicy=first-party-jwt
-n foo10

image pods foo-v1 foo-v2 now direct to istiod-canary,but ingressgateway always direct to istiod

Version

istio:
client version: 1.8.1
control plane version: 1.8.1
data plane version: 1.8.1

kubernetes:
Client Version: v1.18.3
Server Version: v1.18.3

Additional Information

is there something wrong with the code

func GetPilotSan(discoveryAddress string) string { discHost := strings.Split(discoveryAddress, ":")[0] // For local debugging - the discoveryAddress is set to localhost, but the cert issued for normal SA. if discHost == "localhost" { discHost = "istiod.istio-system.svc" } return discHost }

liyuntian avatar May 05 '22 09:05 liyuntian

some error logs: 2022-05-05T08:50:56.712278Z info JWT policy is first-party-jwt 2022-05-05T08:50:56.712285Z info Pilot SAN: [istiod.istio-system.svc] 2022-05-05T08:50:56.712289Z info CA Endpoint istiod-canary.istio-system.svc:15012, provider Citadel 2022-05-05T08:50:56.712315Z info Using CA istiod-canary.istio-system.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2022-05-05T08:50:56.712440Z info citadelclient Citadel client using custom root cert: istiod-canary.istio-system.svc:15012 2022-05-05T08:50:56.743645Z info ads All caches have been synced up in 34.356396ms, marking server ready 2022-05-05T08:50:56.743871Z info sds SDS server for workload certificates started, listening on "./etc/istio/proxy/SDS" 2022-05-05T08:50:56.743894Z info xdsproxy Initializing with upstream address "istiod.istio-system.svc:15012" and cluster "Kubernetes" 2022-05-05T08:50:56.744104Z info sds Start SDS grpc server 2022-05-05T08:50:56.744142Z info Opening status port 15020 2022-05-05T08:50:56.785460Z info Starting proxy agent 2022-05-05T08:50:56.785480Z info Epoch 0 starting 2022-05-05T08:50:56.787146Z info Envoy command: [-c etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --drain-strategy immediate --parent-shutdown-time-s 60 --service-cluster istio-ingressgateway --service-node router~172.19.231.116~istio-ingressgateway-5f58896d5-68tfh.foo10~foo10.svc.cluster.local --local-address-ip-version v4 --bootstrap-version 3 --disable-hot-restart --log-format %Y-%m-%dT%T.%fZ %l envoy %n %v -l warning --component-log-level misc:error] 2022-05-05T08:50:56.935961Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 10.233.0.10:53: no such host" 2022-05-05T08:50:57.065350Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 10.233.0.10:53: no such host" 2022-05-05T08:50:57.164888Z info cache generated new workload certificate latency=421.006365ms ttl=23h59m59.835122854s 2022-05-05T08:50:57.164912Z info cache Root cert has changed, start rotating root cert 2022-05-05T08:50:57.164929Z info ads XDS: Incremental Pushing:0 ConnectedEndpoints:0 Version: 2022-05-05T08:50:57.164958Z info cache returned workload trust anchor from cache ttl=23h59m59.835044398s 2022-05-05T08:50:58.035648Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 10.233.0.10:53: no such host" 2022-05-05T08:50:58.835535Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 10.233.0.10:53: no such host" 2022-05-05T08:51:02.541883Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 10.233.0.10:53: no such host" 2022-05-05T08:51:07.035493Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 10.233.0.10:53: no such host" 2022-05-05T08:51:20.469399Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 10.233.0.10:53: no such host" 2022-05-05T08:51:25.708019Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp 127.0.0.1:15090: connect: connection refused 2022-05-05T08:51:26.146691Z error failed scraping envoy metrics: error scraping http://localhost:15090/stats/prometheus: Get "http://localhost:15090/stats/prometheus": dial tcp 127.0.0.1:15090: connect: connection refused 2022-05-05T08:51:26.860711Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 10.233.0.10:53: no such host" 2022-05-05T08:51:33.257224Z warning envoy config StreamAggregatedResources gRPC config stream closed: 14, connection error: desc = "transport: Error while dialing dial tcp: lookup istiod.istio-system.svc on 10.233.0.10:53: no such hos

liyuntian avatar May 05 '22 09:05 liyuntian

🚧 This issue or pull request has been closed due to not having had activity from an Istio team member since 2022-05-05. If you feel this issue or pull request deserves attention, please reopen the issue. Please see this wiki page for more information. Thank you for your contributions.

Created by the issue and PR lifecycle manager.

istio-policy-bot avatar Aug 19 '22 05:08 istio-policy-bot

Hi @liyuntian, were you able to solve the problem?

admsarabia avatar Jan 13 '23 02:01 admsarabia