Log diff screenshots, so that reporters could add screenshot.

[marcog83]

Hi, thanks for the great tool you created. I started using it in my project. Next step i'd like to do is integrating Allure reporter. Allure creates test logs starting from wdio logs. (COMMAND,RESULT,DATA). When test fails I can see what happened, but I can't retrieve the diff PNG. I think that it's because PNG diff is not logged. It would be awesome if you can add this kind of feature, or if you have some suggestions on how I could add myself.

Thanks marco

this is how my terminal logs everything

this is how allure shows screenshots in logs

[zinserjan]

Sounds really interesting. Unfortunately I can't help. I dont' use allure and have no idea how it works.

Maybe @christian-bromann can help us.

[prayerslayer]

I wrote a custom reporter for this that checks the filesystem for diff images. It's not pretty, but does the job. To be honest I don't know the exact boundaries of different tools involved in this process, so I can't really suggest anything, but having additional information available in the test results would be awesome :)

Relevant part of the reporter:

// Inside CustomReporter.js
this.on('end', function(result) {
    const testcases = [];
    const REFERENCE_DIR = path.resolve(screenshotPath, './reference');
    const referenceFiles = fs.readdirSync(REFERENCE_DIR);
    for (const file of referenceFiles) {
      const diffExists = fs.existsSync(
        path.resolve(screenshotPath, './diff/', file)
      );
      testcases.push({
        failed: diffExists,
        title: file,
        filename: file
      });
    }
  });

[texttechne]

Thanks for your contribution. Can you open up a PR? Then we (whoever that is :-) ) can work together to refine your approach.

[prayerslayer]

Can you open up a PR?

Uhm, which content were you thinking of? 😅 I'm asking because I don't even know if it's a problem of this library or webdriverio in general.

[emilyrohrbough]

+1 This would be great if we could write out the screenshot differences for when isSameDimensions = false. This would add a clear indication that the screenshot received is different, even if the mismatch is zero.

For example, There have been instances where the new image grabs a larger screenshot, i.e. the sizes do not match, however the mismatch is zero. This caused failures within our test setup (as it should), however there is no clear indication as to what caused it when searching the diffs output.

[woolter]

Hi @marcog83 with this you can add the image to allure report:

let allureReport = require('wdio-allure-reporter');
let base64Img = require('base64-img');

exports.config = {
 ...
  afterCommand: function(commandName, args, result, error) {
    if (result && result[0] && result[0].misMatchPercentage) {
      let imageTo64 = base64Img
        .base64Sync(`screenshots/diff/${fileName}`)
        .replace('data:image/png;base64,', '');
      allureReport.createAttachment(
        'Diff image',
        Buffer.from(imageTo64, 'base64'),
        'image/png'
      );
    }
  },
};