Automatically run e2e tests against staging and production after automatically deploying them

[sarayourfriend]

Problem

It would be nice to be able to run the e2e tests automatically against staging and production so that we can absolutely verify that our expectations are reproducible in the live environments and not just in the locally run docker build. This is just a redundancy check.

Description

We need to update the playwright.config.ts to allow passing the base url as an environment variable. We also need to be able to disable the web server option via environment variables.

Then we can run just the e2e tests with BASE_URL=https://search-staging.openverse.engineering SERVER=false pnpm test:playwright e2e.

Additionally, if the e2e tests fail, we can automatically rollback production to the previous release! This part may not yet be possible due to flakiness introduced by the latency of making actual network requests for the e2e tests, so we can table that part for now.

We should also ping slack with the results of the e2e test run for each deployment.

Implementation

• [ ] 🙋 I would be interested in implementing this feature.