feat: Flake Indicator ACI

[mike-plummer]

• Closes #22656 , #22657

User facing changelog

• Display a "flaky test" indicator in the Specs Explorer for specs that are known to be flaky in the Cypress Dashboard

Additional details

• Building new data adapter component to manage fetch/re-fetch of data for the tooltip
• Lots of cloud data management logic got added to SpecsList during initial ACI work - I took this opportunity to extract it out to a new composable. This helps simplify SpecsList so it can move back towards being a presentational component and helps organize some of the cloud spec data flows.
• Adding support for RemoteFetchable* types in our mock gql layer so we can leverage these in tests (like in mountFragment)

Steps to test

1. Download and extract this example project: flaky-e2e.zip
2. Open project in the app - ensure you are logged out
3. Verify that Specs Explorer does not display Flake Indicator, no user-noticeable changes to the UI
4. Log in to the app with your Staging Dashboard account, ACI data should populate
5. Verify that multiple specs are marked with a "Flaky" indicator (note - if other devs have recorded new runs as part of their testing the number of specs marked as flaky may change)
6. Mouse over Flaky indicator, verify tooltip opens and displays data re: flaky state of the test
   • If you want to validate the data you can review the project in the Staging Dashboard
7. Verify that clicking on the indicator or on the tooltip content opens the dashboard to the flaky data section
8. Extra credit: Initiate a new recording from CLI, verify that the app refreshes to reflect in-progress and final data once the recording is complete (approx 30 second polling interval). Verify that flaky tooltips update with new data.

How has the user experience changed?

https://user-images.githubusercontent.com/11482842/182251591-1e20f5d9-7b57-4e7e-a34c-0d656d055079.mov

PR Tasks

• [x] Have tests been added/updated?
• [ ] Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
• [na] Has a PR for user-facing changes been opened in cypress-documentation?
• [na] Have API changes been updated in the type definitions?

[cypress-bot[bot]]

Thanks for taking the time to open a PR!

• Create a Draft Pull Request if your PR is not ready for review. Mark the PR as Ready for Review when you're ready for a Cypress team member to review the PR.
• Become familiar with the Code Review Checklist for guidelines on coding standards and what needs to be done before a PR can be merged.

[cypress[bot]]

---

Test summary

37974 • 0 • 469 • 0 • 4

---

Run details

Project	cypress
Status	Passed
Commit	cdc08bed92
Started	Aug 15, 2022 2:44 PM
Ended	Aug 15, 2022 3:06 PM
Duration	22:21 💡
OS	Linux Debian - 11.3
Browser	Multiple

View run in Cypress Dashboard ➡️

---

Flakiness

	cypress/proxy-logging.cy.ts		2
1	Proxy Logging > request logging > xhr log has response body/status code when xhr response is logged second
2	Proxy Logging > request logging > xhr log has response body/status code when xhr response is logged second
	e2e/origin/commands/assertions.cy.ts		1
1	cy.origin assertions > #consoleProps > .should() and .and()
	create-react-app.cy.ts		1
1	Working with cra-ejected > should detect new spec

---

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

[lmiller1990]

It works!

I have some UI notes (although I know you didn't design the UI, just implement it):

1. It was not immediately obvious I could hover the "flaky" badge
2. I had to hover for around 500ms - I think the tooltip should render faster, if not instantly, to make this more discoverable. Right now this feature requires to to hover a tooltip that isn't obviously hover-able, then wait for quite a bit of time (500ms? longer than a normal tooltip) to render.

I think the badge should change color when I hover it - maybe go darker. Or, the tooltip should render immediately (no delay).

I will look at the code now.

[mike-plummer]

1. It was not immediately obvious I could hover the "flaky" badge

Will pass this along to UX and get their thoughts

2. I had to hover for around 500ms - I think the tooltip should render faster, if not instantly, to make this more discoverable. Right now this feature requires to to hover a tooltip that isn't obviously hover-able, then wait for quite a bit of time (500ms? longer than a normal tooltip) to render.

This has been tricky - originally I had the tooltip pop immediately and display a default state while we fetched data, but that caused a render flicker that wasn't super attractive. Second approach was to pop immediately but render a loading spinner during fetch, but it seems like our Tooltip library doesn't handle child component swapouts very well because it caused the tooltip to jump around briefly which looked even worse (seemed like the Tooltip was trying to auto-place itself again despite size/shape remaining constant). So this was the third solution - delay rendering tooltip until data is fetched which appeared fine on my local but I imagine Australia has higher latency to staging that I do 😆 . I'll take another look because I agree that is too long of a wait - maybe I missed something in the Tooltip or CSS structure that will strike a balance

[lmiller1990]

Re: latency - yes I think the issue is I'm almost as far as geographically possible and my internet is not exactly the fastest. If it feels good on your end, that's fine.

[cypress-bot[bot]]

Released in 10.5.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to Cypress v10.5.0, please open a new issue.