material-ui icon indicating copy to clipboard operation
material-ui copied to clipboard

[test] Update font-awesome CSS file in regression tests fixture

Open Janpot opened this issue 1 year ago • 4 comments

Attempt at stabilizing the icons test. It looks like the referenced css has gone out of sync with the regression tests font loader.

Let's see if this does the trick.

Potentially closes https://github.com/mui/material-ui/pull/43684

Janpot avatar Sep 13 '24 12:09 Janpot

Netlify deploy preview

https://deploy-preview-43745--material-ui.netlify.app/

Bundle size report

No bundle size changes (Toolpad) No bundle size changes

Generated by :no_entry_sign: dangerJS against 25d52a4305415775b8e2923a0d76cac27964d0f2

mui-bot avatar Sep 13 '24 12:09 mui-bot

cc @DiegoAndai who's working on https://github.com/mui/material-ui/pull/43684

aarongarciah avatar Sep 13 '24 12:09 aarongarciah

@Janpot, thanks for taking this!

To clarify, AFAIU, the flaky tests are the Pigment CSS regression ones added here: https://github.com/mui/material-ui/pull/43280

On the pigment fixtures, we have some cases of the icons disappearing: https://app.argos-ci.com/mui/material-ui/builds/31895/107984073

And some with the icons appearing: https://app.argos-ci.com/mui/material-ui/builds/31900/108029355

So, my thought was that network latency was involved. Is that not the case?

DiegoAndai avatar Sep 13 '24 17:09 DiegoAndai

On the pigment fixtures, we have some cases of the icons disappearing: https://app.argos-ci.com/mui/material-ui/builds/31895/107984073 And some with the icons appearing: https://app.argos-ci.com/mui/material-ui/builds/31900/108029355 So, my thought was that network latency was involved. Is that not the case?

@DiegoAndai It seems to match with this PR. We were not waiting for the icon font to load before taking the screenshot.

oliviertassinari avatar Sep 13 '24 18:09 oliviertassinari

So, my thought was that network latency was involved. Is that not the case?

Yes, there's a race condition between the font loading and the moment the screenshot is captured. I noticed there was setup code in the tests that waits until all fonts are loaded before actually capturing the screenshot. Only, it seems like it's waiting for a different CSS file than the one referenced in the test.

I didn't deeply test this, I just corrected the faulty url. It looks like the screenshot contains the icons, so it worked at least once. If this doesn't fix the flakeyness, I plan to debug this fully, it just didn't feel justified to spend more time on it at the moment since this is a clear bug that could explain the flakeyness, with low risk of breakage, we will notice soon enough if it didn't work.

Janpot avatar Sep 16 '24 09:09 Janpot

Seems to not have worked https://app.argos-ci.com/mui/material-ui/builds/32164/109227549

Janpot avatar Sep 16 '24 16:09 Janpot

@Janpot, my take is that we should remove them.

DiegoAndai avatar Sep 16 '24 18:09 DiegoAndai

@Janpot, my take is that we should remove them.

🙂 Nah, let's just fix it

Janpot avatar Sep 17 '24 05:09 Janpot