higress
higress copied to clipboard
helm部署的higress 1.4.2版本升级到v2.0.0-rc.1时higress-controller一直报错Not ready to switch validation to fail-closed
Ⅰ. Issue Description
Helm部署的higress 1.4.2版本,为了解决长链场景下批量添加/修改ingress引起的内存不释放问题,希望升级到higress 2.0版本,按照步骤升级到v2.0.0-rc.1后higress controller疯狂报错。
- 操作步骤
- 1、更新CRD资源
kubectl apply -f https://raw.githubusercontent.com/alibaba/higress/v2.0.0-rc.1/helm/core/crds/customresourcedefinitions.gen.yaml
- 2、因为看到helm repo没有更新版本,就手动更改了higress controller 、 higress gateway的image tag为v2.0.0-rc.1,higress controller启动成功后便一直报错
2024-09-11T08:27:59.009727Z info validationController Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
2024-09-11T08:27:59.022513Z info validationController Successfully updated validatingwebhookconfiguration istio-validator-istio-system (failurePolicy=Ignore,resourceVersion=1016868101)
2024-09-11T08:27:59.022544Z error controllers error handling istio-validator-istio-system, retrying (retry count: 1): webhook is not ready, retry controller=validation
2024-09-11T08:27:59.024060Z info validationController Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
2024-09-11T08:27:59.034137Z info validationController Successfully updated validatingwebhookconfiguration istiod-default-validator (failurePolicy=Ignore,resourceVersion=1016868102)
2024-09-11T08:27:59.034183Z error controllers error handling istiod-default-validator, retrying (retry count: 1): webhook is not ready, retry controller=validation
2024-09-11T08:27:59.035834Z info validationController Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
2024-09-11T08:27:59.035857Z info validationController validatingwebhookconfiguration istio-validator-istio-system (failurePolicy=Ignore, resourceVersion=1016868101) is up-to-date. No change required.
2024-09-11T08:27:59.035877Z error controllers error handling istio-validator-istio-system, retrying (retry count: 2): webhook is not ready, retry controller=validation
2024-09-11T08:27:59.037174Z info validationController Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
2024-09-11T08:27:59.037194Z info validationController validatingwebhookconfiguration istiod-default-validator (failurePolicy=Ignore, resourceVersion=1016868102) is up-to-date. No change required.
2024-09-11T08:27:59.037208Z error controllers error handling istiod-default-validator, retrying (retry count: 2): webhook is not ready, retry controller=validation
2024-09-11T08:27:59.044542Z info validationController Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
2024-09-11T08:27:59.051794Z info validationController Successfully updated validatingwebhookconfiguration istio-validator-istio-system (failurePolicy=Ignore,resourceVersion=1016868104)
2024-09-11T08:27:59.051820Z error controllers error handling istio-validator-istio-system, retrying (retry count: 3): webhook is not ready, retry controller=validation
2024-09-11T08:27:59.052880Z info validationController Not ready to switch validation to fail-closed: dummy invalid rejected for the wrong reason: gateways.networking.istio.io is forbidden: User "system:serviceaccount:higress-test:higress-controller" cannot create resource "gateways" in API group "networking.istio.io" in the namespace "higress-test"
2024-09-11T08:27:59.052895Z info validationController validatingwebhookconfiguration istio-validator-istio-system (failurePolicy=Ignore, resourceVersion=1016868104) is up-to-date. No change required.
2024-09-11T08:27:59.052903Z error controllers error handling istio-validator-istio-system, retrying (retry count: 4): webhook is not ready, retry controller=validation
Ⅵ. Environment:
- Higress version: 1.4.2
- 集群内安装过istio,存在istio-validator
$ kubectl get ValidatingWebhookConfiguration | grep istio
istio-validator-istio-system 1 519d
istiod-default-validator 1 614d