aqueduct icon indicating copy to clipboard operation
aqueduct copied to clipboard

This PR adds regression testing for version upgrades

Open HarisChoudhary opened this issue 3 years ago • 1 comments

We add a github actions that perform validation of workflows against old and new releases. The steps performed for validation are similar what laid out in this page:

https://www.notion.so/aqueducthq/Runbook-Cutting-an-OSS-Release-b72400b7584845f0b0b4ff84bab4aedc#186142d8613f4740b620c90b18e41fee

We take screenshots of the workflow via a browser to simulate the user clicks on the frontend and upload them as screenshots. We also perform a simple validation of the flow through the api client.

Sample actions run: https://github.com/aqueducthq/aqueduct/actions/runs/2862378175

Related issue number (if any)

Checklist before requesting a review

  • [x] I have created a descriptive PR title. The PR title should complete the sentence "This PR...".
  • [x] I have performed a self-review of my code.
  • [x] I have included a small demo of the changes. For the UI, this would be a screenshot or a Loom video.
  • [x] If this is a new feature, I have added unit tests and integration tests.
  • [x] I have run the integration tests locally and they are passing.
  • [x] I have run the linter script locally (See python3 scripts/run_linters.py -h for usage).
  • [x] All features on the UI continue to work correctly.
  • [x] Added one of the following CI labels:
    • run_integration_test: Runs integration tests
    • skip_integration_test: Skips integration tests (Should be used when changes are ONLY documentation/UI)

HarisChoudhary avatar Aug 15 '22 16:08 HarisChoudhary

I will create a demo video and linking it to the notion page where we have the process outline for releases. I think I'll do it after checkin becuase that would allow me to show the start correctly where we can dispatch the workflow from GitHub actions page instead of trigger on branch push.

HarisChoudhary avatar Aug 17 '22 19:08 HarisChoudhary