pipelines
pipelines copied to clipboard
Published
20 hours ago •
kubeflow
kubeflow
[frontend] Flaky tests in CompareV2 resulting in false failures in PRs
Tests in CompareV2.test.tsx seem to be very Flaky, and keep resulting in random failures in PRs, I constantly have to re-run these tests to get it fully passing. Some examples of failure errors I've seen:
failures.1
Summary of all failing tests
FAIL src/pages/CompareV2.test.tsx (91.424 s)
● CompareV2 › Parameters and Scalar metrics tab initially enabled with loading then error, and switch tabs
thrown: "Exceeded timeout of 5000 ms for a test.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."
410 | });
411 |
> 412 | it('Parameters and Scalar metrics tab initially enabled with loading then error, and switch tabs', async () => {
| ^
413 | const getRunSpy = jest.spyOn(Apis.runServiceApiV2, 'getRun');
414 | runs = [newMockRun(MOCK_RUN_1_ID), newMockRun(MOCK_RUN_2_ID), newMockRun(MOCK_RUN_3_ID)];
415 | getRunSpy.mockImplementation((id: string) => runs.find(r => r.run_id === id));
at src/pages/CompareV2.test.tsx:412:3
at Object.<anonymous> (src/pages/CompareV2.test.tsx:34:1)
Test Suites: 1 failed, 113 passed, 114 total
Tests: 1 failed, 1611 passed, 1612 total
Snapshots: 427 passed, 427 total
Time: 238.994 s
failures.2
● CompareV2 › Parameters and Scalar metrics tab initially enabled with loading then error, and switch tabs
Unable to find an element with the text: An error is preventing the Scalar Metrics from being displayed.. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.
<body>
....
OMITTED
....
426 | screen.getByText('There are no Parameters available on the selected runs.');
427 | screen.getByText('An error is preventing the Scalar Metrics from being displayed.');
428 |
at waitForWrapper (node_modules/@testing-library/react/node_modules/@testing-library/dom/dist/wait-for.js:173:27)
at Object.<anonymous> (src/pages/CompareV2.test.tsx:425:18)
Examples:
Love this idea? Give it a 👍.
