rubyforgood
de-flake Cypress tests
Fairly often, cypress fails when nothing is wrong. Make it not do that.
Examples: https://github.com/rubyforgood/casa/pull/2595/checks?check_run_id=3616639951
How it runs in CI: https://github.com/rubyforgood/casa/blob/main/.github/workflows/cypress.yml
How to run it locally:
in 1 window, run RAILS_ENV=test bundle exec rails server -b 0.0.0.0 -p 4040 -d
in another window, run npm run test:cypress
On my local I am having login troubles but those are not the same as on CI as far as I can tell...
Not picking this up at the moment but I have some notes! What we are seeing is different than the listed example, but it can still be filed under "flakey cypress issues"
For the Power hackathon, we've been seeing a lot of network issues. Example - on my client side password validation pr and the birthday presence pr we'd get the following consistently when running ci
1) Logging into cypress as a volunteer
"before all" hook for "should edit the profile":
CypressError: `cy.visit()` failed trying to load:
http://0.0.0.0:4040/
We attempted to make an http request to this URL but the request failed without a response.
We received this error at the network level:
> Error: ESOCKETTIMEDOUT
Common situations why this would fail:
- you don't have internet access
- you forgot to run / boot your web server
- your web server isn't accessible
- you have weird network configuration settings on your computer
This seems like the first test is executing before rails is finished setting up the server, be it webpack isn't done yet or something else... I don't think there's anything that yall can update in the cypress workflow file that would mitigate it, but I'm not versed whatsoever in your ci configuration 😅
A temporary fix could be to add either timeouts to the cypress tests themselves, or add in some retries to the cypress.json file in root like suggested here.
Hope this helps!
