Perform visual regression upon a page elements rather than arbitrary slices of the page

[cmck]

Currently with the option --phantomCSS we split the webpage screenshot into blocks of arbitrary size and perform a visual regression test against those, however this is not robust. See https://github.com/Huddle/PhantomCSS#best-practices. https://github.com/Huddle/PhantomCSS#full-page-screenshots-are-a-bad-idea Ideally we want to compare small sections of a page. Logically it makes sense to select these sections, or 'slices' by CSS selector.

Proposed approach:

1. Compile a list of the CSS selectors for major page elements, eg: header, footer.
2. Obtain the absolute position, height, width of each element
3. Obtain baseline full-page screenshots from Browserstack
4. Slice the screenshots according to the positional data
5. Test: Repeat steps 3-4 and perform PhantomCSS visual regression test

Questions to address:

• can browserstack screenshots API support what we want to do?
• Each browser renders the page slightly differently, does this matter when we have absolute x,y coords?
• Is it trivial to automatically identify all major page elements rather than doing this process manually?

[amitu]

1. Screenshot api cant. It can take screenshots, but can not find out coordinates for DOM nodes. Automate may be used for that.
2. Instead of doing regression testing across different browsers, it would be better done across screenshots from same browser. Eg IE7 yesterday vs today. Across browser does not scale at all as IE6 and iPhone 5 are going to render the same page very drastically differently, no matter what (eg just because lets say browser dimensions are different).
3. No. User have to compile a list of nodes/css selectors they are interested in.