openverse
openverse copied to clipboard
Playwright tests do not fail if the snapshot name is completely new
Description
If the snapshot does not exist in the repository, and is created during the CI run, the CI passes. This happens because we only check if any existing binary files changed, not if there are any new binary files.
Reproduction
- See this run: https://github.com/WordPress/openverse/actions/runs/10851740459/job/30116308306?pr=4914