Flaky E2E tests identified in 1.9.x

[squakez]

We may have a look and collect here the tests that are mostly failing in our E2E suite:

• [ ] --- FAIL: TestKameletChange (269.36s)
• [ ] --- FAIL: TestKitKnativeFullBuild (619.67s)

[squakez]

@phantomjinx @tadayosi @oscerd @astefanutti please, use this issue to add any repeated failure you may experience (I mean, anything that use to happen ofter enough to be considered really flaky).

[phantomjinx]

Tests that kept failing in the OCP4 run of the test are 'labelled' with the CAMEL_K_TEST_SKIP_PROBLEMATIC env var so that test runs can skip (nightly OCP4 runs do this at the moment).

• [ ] --- FAIL: TestMetrics (reconciliation duration metric)
• [ ] --- FAIL: TestRunDevMode
• [ ] --- FAIL: TestTektonLikeBehaviour
• [ ] --- FAIL: TestRunRoutes
• [x] --- FAIL: TestHealthTrait
• [ ] --- FAIL: TestOpenAPI
• [ ] --- FAIL: TestOpenAPIConfig
• [ ] --- FAIL: TestMasterTrait
• [ ] --- FAIL: TestTolerationTrait

[tadayosi]

Right now I only remember this local e2e test, which hangs up without exiting and timeouts the entire test.

• [ ] --- FAIL: TestLocalRunIntegrationDirectory

[squakez]

--- FAIL: TestKameletChange (269.36s) will be fixed by #3256

[tadayosi]

• [x] --- FAIL: TestHealthTrait -- fixed by #3310

[tadayosi]

This one also becomes flaky with openshift workflow.

• [ ] --- FAIL: TestKitTimerToLogFullNativeBuild (1219.43s)

[tadayosi]

With recent efforts including #3478 E2E has become more stable now. We can check again if those listed above are still flaky or improved already.

[tadayosi]

Maybe we can close this for now. Our E2E is stable, except a couple flaky ones which are mostly for native builds and due to resource limitations of GitHub action runners.

[squakez]

Yes, the latest merges (ie, 821ef90a5175a998df971e19a25f9526fc01ef3b) are not reporting any error.