eui icon indicating copy to clipboard operation
eui copied to clipboard

Configure Loki to test open PRs in CI

Open tkajtoch opened this issue 1 year ago • 3 comments

Summary

Loki is a Visual Regression Testing software that integrates with Storybook. We need to run it in CI for every open PR to ensure no unwanted UI changes get merged to main.

Please follow the guide on Loki's website: https://loki.js.org/continuous-integration.html.

Due to the architecture of our current CI pipeline, Loki CI step should depend on the eui-pull-request-deploy-docs step since it depends on Storybook being deployed.

Acceptance Criteria

  • Loki should run as a separate CI step after Storybook is built and deployed
  • loki command should be executed with the --requireReference flag and CI=true environment variable set
  • failed tests should result in a failed step status and blocked PR merge

tkajtoch avatar Mar 25 '24 15:03 tkajtoch

The blocking issue has just been merged

tkajtoch avatar Apr 29 '24 15:04 tkajtoch

Will be unblocked and ready for review when https://github.com/elastic/ci-agent-images/pull/691 is merged

tkajtoch avatar May 20 '24 11:05 tkajtoch

I moved this to on hold since there's a race condition in Loki that seems to happen randomly when running in CI with Storybook 8.

For more context: https://github.com/storybookjs/test-runner/issues/442

tkajtoch avatar May 28 '24 11:05 tkajtoch