element-web icon indicating copy to clipboard operation
element-web copied to clipboard

[Disabled] Flaky playwright test: `Timeline › message displaying > should set inline start padding to a hidden event line`

Open richvdh opened this issue 1 year ago • 7 comments

https://github.com/matrix-org/matrix-react-sdk/actions/runs/7559429855:

 1) [Rust Crypto] › timeline/timeline.spec.ts:555:13 › Timeline › message displaying › should set inline start padding to a hidden event line 
    Error: Screenshot comparison failed:

      28 pixels (ratio 0.01 of all image pixels) are different.

    Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/timeline/timeline.spec.ts/hidden-event-line-zero-padding-irc-layout-linux.png
    Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/timeline-timeline-Timeline-message-displaying-should-set-inline-start-padding-to-a-hidden-event-line-Rust-Crypto/hidden-event-line-zero-padding-irc-layout-actual.png
        Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/timeline-timeline-Timeline-message-displaying-should-set-inline-start-padding-to-a-hidden-event-line-Rust-Crypto/hidden-event-line-zero-padding-irc-layout-diff.png

    Call log:
      - page._expectScreenshot with timeout 5000ms
      -   verifying given screenshot expectation
      - waiting for locator('.mx_MainSplit')
      -   locator resolved to <div class="mx_MainSplit">…</div>
      - taking element screenshot
      -   disabled all CSS animations
      - waiting for fonts to load...
      - fonts loaded
      -   waiting for element to be visible and stable
      -   element is visible and stable
      - 28 pixels (ratio 0.01 of all image pixels) are different.
      - waiting 100ms before taking screenshot
      - waiting for locator('.mx_MainSplit')
      -   locator resolved to <div class="mx_MainSplit">…</div>
      - taking element screenshot
      -   disabled all CSS animations
      - waiting for fonts to load...
      - fonts loaded
      -   waiting for element to be visible and stable
      -   element is visible and stable
      - captured a stable screenshot
      - 28 pixels (ratio 0.01 of all image pixels) are different.


      585 |             };
      586 |
    > 587 |             await expect(page.locator(".mx_MainSplit")).toMatchScreenshot(
          |                                                         ^
      588 |                 "hidden-event-line-zero-padding-irc-layout.png",
      589 |                 screenshotOptions,
      590 |             );

        at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/timeline/timeline.spec.ts:587:57

Looking at the screenshot, the room avatar appears to be an "E" rather than a "T":

Diff:

image

image

richvdh avatar Jan 18 '24 11:01 richvdh

2nd strike: https://github.com/element-hq/element-web/actions/runs/7639565834

t3chguy avatar Jan 24 '24 12:01 t3chguy

This is https://github.com/element-hq/element-web/issues/22590

t3chguy avatar Jan 24 '24 12:01 t3chguy

I'm not sure disabling this test will do much good, given the failure mode can happen to any test

t3chguy avatar Jan 24 '24 12:01 t3chguy

3rd strike: https://github.com/element-hq/element-web/actions/runs/7744210809

MidhunSureshR avatar Feb 02 '24 06:02 MidhunSureshR

@MidhunSureshR: I updated the summary.

richvdh avatar Feb 03 '24 14:02 richvdh

This flaky test issue has not been updated in 14 days. It is being closed as presumed resolved.

github-actions[bot] avatar Apr 23 '24 01:04 github-actions[bot]

This flaky test issue has not been updated in 14 days. It is being closed as presumed resolved.

github-actions[bot] avatar May 17 '24 01:05 github-actions[bot]