restricted-site-access
restricted-site-access copied to clipboard
Inconsistent test results between local and Github Actions environment
Describe the bug
Tests pass locally but fail on GitHub actions. So far the cause is unknown.
- Tests pass for
WP:latest
but fail forWP:trunk
andWP:mininmum
- This issue was first detected in PR #169 at specifically this line.
- This is the GitHub Actions log pointing to the test failure.
- The only way to make the tests pass was to rewrite the test with an alternative approach.
What has been done so far to debug this?
With @dinhtungdu's suggestion, we debugged every test case with the help of Cypress's screenshot and video recording features, which were uploaded using GitHub Actions artifacts.
This is the video artifact that shows the failure on Github Actions:
https://user-images.githubusercontent.com/17757960/145206299-def50e06-3904-4c2c-93ee-317e477c2838.mp4
It is clear from the video that redirecting to /accessible-page
returns a 404 despite the page being published.
Steps to Reproduce
- Fork this repo
- After setting up the local environment, modify the test cases to the failing state as seen in PR #169
- Run
npm run wp-env start && npm run create-pages
- Run
npm run cypress:run
- See that all test cases pass
- Commit and push and check GitHub actions, 1 test fails
Expected behavior Cypress behavior should be consistent between local and CI environments.
@Sidsector9 did #171 resolve this?
Jeff PR #171 seem to be unrelated to this issue. @mitogh can you please confirm?
Closing this as it is an edge case. Will reopen if the issue reoccurs.