simorgh icon indicating copy to clipboard operation
simorgh copied to clipboard

Integration test investigation

Open karinathomasbbc opened this issue 4 months ago • 3 comments

Resolves JIRA [number]

Overall changes

Code changes

Testing

Things I've tried

  • [x] describe.only - this causes tests to fail as expected but is not acceptable as a long term solution
  • [x] yarn test:integration —runInBand - already running in band
  • [x] Delete all tests except for homePage (again not long term solution) - some tests failures did occur, but not the expected ones
  • [ ] Delete arabic snapshots (not acceptable as long term solution, but if it "works" and we run out of options, we could look to delete the snapshot files are part of the integration test command.... hmmm that won't work with yarn:integration:ci because the tests will always fail due to no snapshots 🤯 )
  • [x] Fix Error: Uncaught [SecurityError: localStorage is not available for opaque origins] - could this actually be causing multiple failures but is being handled by the retries somehow?
    • [x] this has now been fixed on this PR, but it has not caused the tests to fail as expected

Other ideas (brain dump)

  • remove test retries, and only attempt once (may introduce flakiness / instability?)
  • change max workers
  • sharding
  • run tests sequentially instead of in parallel
  • run with verbose to see what's going on?
  • console logging of fetchDom / document / html
  • remove silent reporter from integration:ci to see if we can view the individual test passes - like we can see for the NextJS integration output. See https://github.com/bbc/simorgh/actions/runs/8017605611/job/21901822157 -> Integration Tests - Next

image

karinathomasbbc avatar Feb 23 '24 08:02 karinathomasbbc