kyma
kyma copied to clipboard
Stabilize API-gateway pipeline
Description After describing API-gateway pipeline with Gherkin it is apparent that API-gateway pipeline is often failing. We need to investigate issues causing it to fail and stabilise the pipeline.
Observed problems The pipeline failed with many different errors, the most prelevant issue is Istio validation webhook context deadline exceed: https://status.build.kyma-project.io/view/gs/kyma-prow-logs/pr-logs/pull/kyma-project_kyma/14670/pre-main-kyma-gardener-gcp-goats-api-gateway/1540235233620660224
This issue also causes API Rule creation and update to fail sometimes.
"status":{
"APIRuleStatus":{
"code":"ERROR",
"desc":"Internal error occurred: failed calling webhook \"[validation.istio.io](http://validation.istio.io/)\": Post \"[https://istiod.istio-system.svc:443/validate?timeout=10s](https://istiod.istio-system.svc/validate?timeout=10s)\": context deadline exceeded"
},
"accessRuleStatus":{
"code":"ERROR"
},
"lastProcessedTime":"2022-07-11T06:45:16Z",
"observedGeneration":2,
"virtualServiceStatus":{
"code":"ERROR"
}
}
Related issues https://github.com/kyma-project/kyma/issues/14473
There is no errors on gardener.
Changed concurrency to 1
Error from server (InternalError): error when creating "/home/prow/go/src/github.com/kyma-project/kyma/ory-hydra-login-consent.yaml": Internal error occurred: failed calling webhook "validation.istio.io": failed to call webhook: Post "https://istiod.istio-system.svc:443/validate?timeout=10s": context deadline exceeded 647 2022-07-29 07:52:03 UTC [ERROR] PID:13 --- [/home/prow/go/src/github.com/kyma-project/test-infra/prow/scripts/lib/gardener/gcp.sh] gardener::cleanup:1 AN ERROR OCCURED! Take a look at preceding log entries.
since some time pipeline seems to be green. issue dummy PR and test all to confirm it's stable