cobalt icon indicating copy to clipboard operation
cobalt copied to clipboard

feat: add `@playwright/test` setup and tests

Open ndom91 opened this issue 2 years ago • 2 comments

Changes

  • Adds testing setup using playwright, a modern headless browser E2E testing framework
  • Adds initial 2 tests to ensure API and Webapp are spun up and reachable
  • Also started copying some of your tests from tests.json definition into playwright tests.
    • I've got a bunch of twitter, as well as 1 soundcloud and 1 youtube
    • This shuold be enough to be able to replicate the rest of them as these all contain examples on how to modify settings in cobalt, listen to request bodies, listen to response bodies, etc. so the rest should be mostly copy/paste and slightly modify versions of these.
    • See the test at the bottom of testPlaywright/providers/twitter.spec.js for one which is fairly complicated, i.e. covers a lot of ground, and is well commented as a good example to build from.
  • Adds Github Action which runs this on every push/PR toward current
  • You can also run npm run playwright:test locally. Ensure that your .env file doesn't ahve anything in it, or just rename it to something else temporarily, as your "web / api mode" selection thing is a bit convoluted. Playwright spins up its own copies of the api and web app whenever you start it, both with the appropriate env vars (see webServers in the playwright.config.js)
  • If a playwright test fails, it dumps a really useful trace (zip file) of all the actions / navigations / etc that it did as well as a video if you want. Traces can be uploaded and viewed here or opened via the command it spits out upon failure: image

Explanation

  • A little explanation about how this works if you're not familiar, so by running the test:playwright npm script, it will start up playwright using the configs found in the new playwright.config.js file. An important part of that is the webServer section toward the bottom. This tells playwright to spin up the API as well as the webapp which we'll be running the tests against.
  • Playwright is first and foremost a headless browser testing utility, with which you can programmatically control browser interaction (click here, type X into Y input, etc.), but they also have robust support for API testing now as well.
    • Getting started generally: https://playwright.dev/docs/writing-tests
    • API Testing: https://playwright.dev/docs/api-testing

ndom91 avatar Dec 19 '23 12:12 ndom91

Also small sidenote, normally you should commit your package-lock.json so everyone gets the exact same version of the dependencies as you have. Without committing the lock file to the repository, everyone's npm will install the latest version of the dependencies allowed by the prefix in their package.json entry. So for example, nanoid: '^4.0.2' says nanoid patch version updates are allowed (defined by the ^ character), so theoreitclaly if theres a 4.0.99 version out there, when I run npm install I would be allowed to get that, where you're still operating with 4.0.2. Know what I mean? This can be avoided by just committing your package-lock.json to the repository :+1:

ndom91 avatar Dec 19 '23 13:12 ndom91

@wukko I would appreciate if you approved the latest run which can be found here, so we can see if everything works on github actions as well. It shuold run without issue :blush: it does locally :crossed_fingers:

ndom91 avatar Dec 19 '23 13:12 ndom91

started copying some of your tests from tests.json definition into playwright tests.

can't you just get the test urls from tests.json instead of hardcoding them?

ihatespawn avatar Jul 07 '24 22:07 ihatespawn

we're rewriting frontend from ground up, so these tests aren't compatible :(

feel free to make new ones after it's merged into current branch!

wukko avatar Jul 08 '24 01:07 wukko