Open-Assistant
Open-Assistant copied to clipboard
LAION-AI
Automate e2e test of the website
At the moment these are enabled for PRs that change the website and for pushes to main. Note it does take 3-10mins to run. Closes #274 (though it only runs the e2e tests and doesn't run the component tests at the moment, but we only have one at the moment anyway).
See https://github.com/othrayte/Open-Assistant/pull/1 and https://github.com/othrayte/Open-Assistant/pull/2 for an example of what a passing and failing PR looks like after this change. Note that you can both see in the log what failed and download images of the failure and videos of the run.
Edit: Please pay particular attention to when the workflow is triggered, what I have is just my initial assumptions and even I'm now thinking it might need changing -othraye.
Additional examples:
- Not triggering if no web or backend change https://github.com/othrayte/Open-Assistant/pull/3
- Triggering on backend change https://github.com/othrayte/Open-Assistant/pull/4
do I see this correctly, this will build & push the images on ever single push to main? because so far we're only doing that on release. I'm fine with building all the time, but is it possible that you also change the release workflow, specifically what needs to happen is that the ansible playbook must take as a parameter the image tag (default latest) that must be deployed, and then the release workflow must pass the release version tag to the ansible playbook. I.e. when we make the release "v0.1.0", then the ansible playbook should deploy v0.1.0, and not latest (as it does now).
let me know if you can do this. if not, we can just merge this and take care of ansible later.
Second possibility: don't push the docker images, but build them locally in the workflow and use them from there
Third possibility: run the e2e tests as a cron job every night
@yk, that's something I didn't consider, though if I'm reading https://github.com/docker/metadata-action#latest-tag correctly the latest tag should not be updated on the push and pr builds, just the release (tag) one. So maybe no changes are required.
If we are happy to have the extra builds there I think it provides a performance benefit as we test the browsers in parallel but use the same docker images. I don't see any issues with passing a specific tag to the ansible job, we even have the tags that are now provided by the docker build workflow that we can pass to ansible to select the exact image we just built (which is how the e2e tests do it).
I've just raised an alternative simpler PR as an option that should have less potential for fallout. Take a look. #439
I assume no-one has any objections to declining this now that we have merged the alternative https://github.com/LAION-AI/Open-Assistant/pull/439
