system-tests
system-tests copied to clipboard
DataDog
Make system tests works on forks ?
idea 1 : distant repo has a valid DD_API_KEY in secrets idea 2: system test repos hosts the process
Some context of why this would be useful:
System tests always fail on forks. This means that we can never know if an external PR breaks something until it's merged. It also means that we cannot require system tests to pass in order to merge, otherwise it blocks merging forks completely.
So, I just learned that a button Approve and run exists :)
https://docs.github.com/en/actions/managing-workflow-runs/approving-workflow-runs-from-public-forks
@cbeauchesne The button exists, and it does appear in PRs, but clicking it only results in system tests running and always failing because it cannot access the API key.
This is also true for CircleCI. Would it be possible to split up running the tests and reporting the results? I know this might not work well with end to end tests, but at least it would be something.
idea 3 : mocked backend
it will work for almost all scenario, as we don't test responses from backend. And will be able to use a fake key.
idea 4 that actually work fine : using a fake DD_API_KEY. Turns out this key does not have to be a real one to have everything working \o/