datahub
datahub copied to clipboard
Switch from running E2E tests against preview deployments to running them against local instance in the workflow (?)
Before running E2E tests in the GitHub workflow, Playwright runs our custom global.setup.ts
file. It's purpose is to sync the test site before running tests against it. So the steps are:
- wait for the branch preview deployment on Vercel to complete
- sync the site (necessary if there has been any changes made to it and our tests depend on those changes) - will update R2 staging bucket and the staging db
- run E2E tests against the preview deployment
There are two problems that can cause E2E tests to fail:
- the site sync performed in step 2 will not use the branch version of the inngest sync workflow, but rather the
staging
version of it; this means if you've made some changes to the workflow itself and you want to run your tests against a site that has been synced with an updated workflow - it won't work - even if the above worked, there is a potential issue related to the site being already "in-sync" with the latest github changes (e.g. some else synced the site, manually or through the E2E workflow on their own PR); this means even if you run your version of the sync workflow, it would return immediately, because the hash of the tree stored in R2 and of the latest one pulled from GitHub would be the same.
It should be possible to solve the first problem with setting branch environments in Inngest.