[Feature] add option to only update snapshots that has been changed

[rikardthulin]

--update-snapshots regenerates all images, but typically we only wan't to update the images that has been detected to be invalid from a previous run.

So an option like --update-failed-snapshots

[mxschmitt]

do you mean something like this?

npx playwright test
-> 999 tests are passing, 1 failing because of a different snapshot
npx playwright test --update-failed-snapshots
-> it will run a single test, the previously failed one and regenerate it's snapshot?

[rikardthulin]

Yes, spot on!

[nowseemee]

here is a rudimental bash script which reads from the existing JSON results and runs the failed tests one by one:

sample=$(jq '[.suites[].specs[] | select(.ok == false) | {name: .title}]' ./playwright-report.json)
for row in $(echo "${sample}" | jq -r '.[] | @base64'); do
  _jq() {
    echo ${row} | base64 --decode | jq -r ${1}
  }
  npx playwright test -g "$(_jq '.name')" -u
done

[kirillantv]

Nice feature to have. Just another one workaround with js script but requires to switch on JSON reporter. Works for me (I am using different projects that generates reports into one folder):

const shell = require('shelljs');
const fs = require('fs');
const path = require('path');

const reportsFolder = './reports'

const testsToUpdate = {}

const files = fs.readdirSync(reportsFolder)

for (fileName of files) {
    const rawReportData = fs.readFileSync(`${reportsFolder}/${fileName}`, {encoding:'utf8', flag:'r'})
    const jsonReportData = JSON.parse(rawReportData)

    for (suite of jsonReportData.suites) {
        for (spec of suite.specs) {
            const title = spec.title

            for (test of spec.tests) {
                const projectName = test.projectName

                for (result of test.results) {
                    if (result.status == 'failed') {
                        if (!(projectName in testsToUpdate)) {
                            testsToUpdate[projectName] = []
                        }

                        testsToUpdate[projectName].push(title)
                    }
                }
            }
        }
    }
}

for (const [projectName, testTitles] of Object.entries(testsToUpdate)) {
    let testsToRun = testTitles.join('|')

    shell.exec(`npx playwright test --update-snapshots --project=${projectName} -g "${testsToRun}"`)
}