camel-k
camel-k copied to clipboard
apache
Flaky E2E tests identified in 1.9.x
We may have a look and collect here the tests that are mostly failing in our E2E suite:
- [ ] --- FAIL: TestKameletChange (269.36s)
- [ ] --- FAIL: TestKitKnativeFullBuild (619.67s)
@phantomjinx @tadayosi @oscerd @astefanutti please, use this issue to add any repeated failure you may experience (I mean, anything that use to happen ofter enough to be considered really flaky).
Tests that kept failing in the OCP4 run of the test are 'labelled' with the CAMEL_K_TEST_SKIP_PROBLEMATIC env var so that test runs can skip (nightly OCP4 runs do this at the moment).
- [ ] --- FAIL: TestMetrics (reconciliation duration metric)
- [ ] --- FAIL: TestRunDevMode
- [ ] --- FAIL: TestTektonLikeBehaviour
- [ ] --- FAIL: TestRunRoutes
- [x] --- FAIL: TestHealthTrait
- [ ] --- FAIL: TestOpenAPI
- [ ] --- FAIL: TestOpenAPIConfig
- [ ] --- FAIL: TestMasterTrait
- [ ] --- FAIL: TestTolerationTrait
Right now I only remember this local e2e test, which hangs up without exiting and timeouts the entire test.
- [ ] --- FAIL: TestLocalRunIntegrationDirectory
This one also becomes flaky with openshift workflow.
- [ ] --- FAIL: TestKitTimerToLogFullNativeBuild (1219.43s)
With recent efforts including #3478 E2E has become more stable now. We can check again if those listed above are still flaky or improved already.
Maybe we can close this for now. Our E2E is stable, except a couple flaky ones which are mostly for native builds and due to resource limitations of GitHub action runners.
Yes, the latest merges (ie, 821ef90a5175a998df971e19a25f9526fc01ef3b) are not reporting any error.