open-saas icon indicating copy to clipboard operation
open-saas copied to clipboard

Improve e2e tests setup/running

Open Martinsos opened this issue 8 months ago • 0 comments

Right now, we have different ways to run e2e tests locally and in CI.

In CI, there are special steps happening just in Github Actions.

For running locally, we ask user to run web app manually and then run e2e tests.

This complicates maintenance of e2e tests setup since we have two workflows we have to care about and that could diverge by accident.

Ideally, we would have a single command or something that would run e2e tests and work both in CI and locally. Or, maybe two commands, one for CI, and one for local, that would share as much of the logic as possible (probably they would call a common comand + some minimal pre-steps or configuration via env vars). And we wouldn't require user to run web app manually locally.

The reason why we have these things separate though is due to the peculiarities of Github Actions and how some parts of e2e tests pipeline, and of wasp processes, behave there. Also, I remember Playwright had some issues with how it reacts to sigkill (doesn't do it well, there is an issue open on their GH). So a couple of things that forced us into this situation. We tried solving it, but solution wasn't easy to reach, so we deffered it for the future.

WHoever is going after this, I suggest you check the existing code for TODOs and NOTEs, and potentially also older PRs even for comments on what we had trouble with.

If we crach this, we can transfer what we learn also back to wasp repo and other wasp examples and templates.

Martinsos avatar Jun 21 '24 13:06 Martinsos