istio.io icon indicating copy to clipboard operation
istio.io copied to clipboard

Gateway API test flakes

Open craigbox opened this issue 1 year ago • 3 comments

https://prow.istio.io/view/gs/istio-prow/pr-logs/pull/istio_istio.io/15483/doc.test.profile-default_istio.io/1825705272044163072

The first failure is in TestDocs/tasks/traffic-management/ingress/ingress-control/gtwapi_test.sh.

Cleanup runs, with output looking similar to non-flaky runs. Then, the failure is that the snapshot differs by the continued presence of httpbin-gateway, which should have been removed in the cleanup.

There's a smelly TODO and sleep 10 in the cleanup.

A half a dozen more tests run and pass, then TestDocs/tasks/traffic-management/mirroring/gtwapi_test.sh runs. It removes its own httpbin HTTPRoute, thus failing its own before/after for the reverse reason.

(Link to prior discussion of similar flakes.)

craigbox avatar Aug 20 '24 03:08 craigbox

I've added fixes (https://github.com/istio/istio.io/pull/15483/commits/11d75fadca83c7ce3ec05894169fc07447264c4e) in https://github.com/istio/istio.io/pull/15483 which have let me pass the tests twice in a row now.

craigbox avatar Aug 21 '24 02:08 craigbox

https://prow.istio.io/view/gs/istio-prow/pr-logs/pull/istio_istio.io/15595/doc.test.multicluster_istio.io/1826821536166711296

TestDocs/setup/install/external-controlplane/gtwapi_test.sh isn't cleaning up the istio-revision-tag-default webhook:

        -      "mutatingWebhookConfigurations": null,
        +      "mutatingWebhookConfigurations": [
        +        "istio-revision-tag-default"
        +      ],

But it also suggests a failure in the test.

craigbox avatar Aug 23 '24 04:08 craigbox

https://github.com/istio/istio/issues/36905 for the webhooks

craigbox avatar Aug 28 '24 09:08 craigbox