swift-snapshot-testing
swift-snapshot-testing copied to clipboard
Add isFailOnNewRecording for automatic recording
I work on Screenshotbot, which is a screenshotbot/snapshot testing service, which automates all of the recording workflow. I'm also the original author of https://github.com/facebook/screenshot-tests-for-android, and previously built something very similar to Screenshotbot at Facebook.
With Screenshotbot, we would want to move the recording flow to CI. So developers shouldn't have to manually record screenshots. They should make the changes, push to CI/open a pull request, and the CI would automatically comment on the PR with whatever changes happened, (or if it's already merged, then sends a Jira/Asana/Github Issue task to the author).
But this requires that the tests don't fail when the snapshots change (because a screenshot change in this flow isn't a failure, it's just something that we send tasks for). And that's the feature I'm adding in this PR. It's a relatively straightforward change, but let me know if you need more information from me.
The test failure seems related to the Swift upgrade. I had to make that change locally in order to run make test-swift
. Let me know if you'd like to fix that in this PR.
I was going to submit something similar and found this PR. Would be great to have this feature 👍
Closing this out, since I'm making the default behavior of my fork incompatible with upstream. (Just to make it easier for Screenshotbot users to integrate with swift-snapshot-testing. i.e. the fork's default behavior will be to not fail when an snapshots don't exist, or if the snapshot is different. I understand that this behavior doesn't make sense for everyone else.)
Forking will also let me keep the behavior different for PNGs vs non-image snapshots, since I want the old behavior for non-images.
Apart from that, I'll keep my fork up-to-date with the latest version of swift-snapshot-testing.