moj-frontend icon indicating copy to clipboard operation
moj-frontend copied to clipboard

Add visual regression tests to components with HTML/CSS content

Open gregtyler opened this issue 4 years ago • 0 comments

Summary

It would be helpful to add visual regression tests to components which introduce new HTML/CSS content. This would allow us to identify unintended regressions and to visually identify the impact of changes. For example, if GDS change variables (like colours) upstream and we bump the dependency, we can ensure it doesn't break our components.

Motivation

When making changes to dependencies or refactoring, it's hard to know if you've visually broken a component without manually assessing them. Even then, it can be hard to spot regressions. And when making changes which you intend to cause visual changes, it can be hard to explain these changes or spot side effects. Providing visual regression information on the PR would alleviate both of these concerns.

Describe alternatives you've considered

The only real alternative I can think of is not doing it to save the effort.

Additional context

There are three parts to this:

  • Taking screenshots of components and storing them somewhere (GitHub? CircleCI?) so you can compare any new screenshots with what's currently in master
  • Finding a tool to produce visual diffs between two screenshots (or report that there isn't any difference)
  • Generating diffs in the PR build process and writing them onto to the PR to be reviewed

BBC had a tool called wraith which produces visual regression diffs but it doesn't seem to still be maintained

gregtyler avatar Jan 08 '21 16:01 gregtyler