Tip: Creating an HTML report

[priomsrb]

EDIT: See kirilldronkin's comment below for updated instructions.

You can easily create a HTML report using reg-cli:

npm install -g reg-cli
cd .loki
reg-cli ./current/ ./reference/ ./reg-diff/ -R ./report.html
cp -r difference/ reg-diff # Use loki's diffs
open report.html

This will generate a report that looks something like this

[ryparker]

Thanks for the tip! Question though, why have reg-cli redo the image diffing if loki already does it? In your script I notice you copy the loki differences into the ref-diff folder AFTER the reg-cli creates its own diffs in ./reg-diff/. Any reason why you wouldn't want to omit the reg-cli diffing and just use the loki diffs to create the report?

[priomsrb]

Hi @ryparker.

I'm not sure if it's possible to prevent reg-cli from doing its own diffing. So I just replace its diffs with Loki's as you mentioned.

[ryparker]

You're right, the html report is only created after the diffing. So perhaps only using reg-suit's diffing would make more sense.

I actually prefer reg-suit's extra features for managing images and notifications. So I only use Loki to get the screenshots of each storybook component. I then use reg-suit to pull images from S3 and let it handle the diffing. Then it creates a nice report in S3, sends slack messages and comments on the PR.

Here is what my script looks like:

#!/usr/bin/env bash

# Need fresh build of storybook to test
yarn build-storybook
# Use loki to take the screenshots of storybook components FAST (No diffing happening here)
loki update --reactUri file:./storybook-static
# Reg-suite will handle the diffing, s3 storage, and notifications (github/slack)
reg-suit run -c ./.storybook/visual-regression-tests/regconfig.json

[stefanteixeira]

@priomsrb I really liked that idea! Loki misses a HTML report ability. But after copying Loki's difference folder to reg-diff, the report remains unchanged (containing only reg-clidiffs). 🤔

Is there an easy way to update the report making it use just the Loki diffs? For now I would prefer to use Loki for diffing, but reg-cli report is awesome indeed.

[JohnAlbin]

Is there an easy way to update the report making it use just the Loki diffs?

I looked at the CLI options for reg-cli and it doesn't look like there is a way to use existing diff images instead of having it generate new diff images. Maybe they'd accept a PR for that? However, reg-cli seems to have more options related to the diffing than loki does. https://github.com/reg-viz/reg-cli#usage

I've been playing with reg-cli and this is the current set of package.json commands I use to have loki generate the current test images and reg-cli to generate the diff images and the report:

    "loki-test": "rm ./.loki/diff/* ./.loki/difference/* ./.loki/current/*; loki test; yarn run loki-test-report",
    "loki-test-report": "reg-cli ./.loki/current ./.loki/reference ./.loki/diff --extendedErrors --enableAntialias --report ./.loki/report.html --json ./.loki/report.json",

Notes: I don't need to run yarn run loki-test-report ever, but separating it into its own command makes it easier to read the package.json. The Loki diff images are stored in .loki/difference in case I want to compare them to the ones that reg-cli puts in .loki/diff.

Shouldn't this tip be put into the docs? This issue is buried on the second page now; I only saw it because I decided to post a bug report before switching to a different visual regression tool.

[kirilldronkin]

reg-cli bypasses diffing when --from option is specified, so we can generate an input json file after running Loki based on testing results:

scripts/generate-visual-json-report.js:

const {promisify} = require('util');
const {readdir, writeFile} = require('fs');
const {join: joinPath, relative} = require('path');

const asyncReaddir = promisify(readdir);
const writeFileAsync = promisify(writeFile);

const lokiDir = joinPath(__dirname, '..', '.loki');
const actualDir = joinPath(lokiDir, 'current');
const expectedDir = joinPath(lokiDir, 'reference');
const diffDir = joinPath(lokiDir, 'difference');

(async function main() {
	const diffs = await asyncReaddir(diffDir);

	await writeFileAsync(joinPath(lokiDir, 'report.json'), JSON.stringify({
		newItems: [],
		deletedItems: [],
		passedItems: [],
		failedItems: diffs,
		expectedItems: diffs,
		actualItems: diffs,
		diffItems: diffs,
		actualDir: relative(lokiDir, actualDir),
		expectedDir: relative(lokiDir, expectedDir),
		diffDir: relative(lokiDir, diffDir)
	}));
})();

package.json:

{
  "scripts": {
      "visual:report": "npm-run-all visual:report:*",
      "visual:report:json": "node scripts/generate-visual-json-report.js",
      "visual:report:html": "reg-cli --from .loki/report.json --report .loki/report.html"
  }
}