[Feature]: ability to save snapshot diffs with trace when --update-snapshots is run

[keyserj]

trafficstars

🚀 Feature Request

With npx playwright test, if a snapshot fails and traces are being saved, the trace will have nice attachments for debugging - actual.png, expected.png, diff.png.

But with npx playwright test --update-snapshots, these attachments are not saved. It would be nice to have these when snapshots are being updated, so that I can still see the pixel-precise differences. Potentially saving these in this case could be based on a config or CLI option.

Initial thread on Discord

Example

No response

Motivation

I'm using a workflow where I assume that snapshot diffs are intentional, so that when there are diffs I don't have to run the tests a second time just to fix them. I'm also running this in CI, and automatically creating a commit with the diffs.

Most of the time, I expect this to be good because the diffs are intentional, and usually using github/vscode git diff to compare before/after is good enough, but sometimes there have been small pixel diffs that are hard to tell without the playwright-generated diff file.

Since I'm using version control, I can always run the tests locally without --update-snapshots in order to see the diffs (or maybe manually use the same algorithm that playwright uses to create a diff file), but that seems like more effort than is necessary.

[Skn0tt]

related to https://github.com/microsoft/playwright/issues/23197, which is also about when to attach actual and diff.

[pavelfeldman]

Why was this issue closed?

Thank you for your involvement. This issue was closed due to limited engagement (upvotes/activity), lack of recent activity, and insufficient actionability. To maintain a manageable database, we prioritize issues based on these factors.

If you have additional information not present in this issue that you think will help prioritizing it, please open a new issue and reference this one. More support or clarity on its necessity may prompt a review. Your understanding and cooperation are appreciated.