maestro icon indicating copy to clipboard operation
maestro copied to clipboard

feat: add new `assertVisual` command

Open TheKohan opened this issue 1 year ago • 7 comments

Proposed changes

This PR is built on top of #1898 by @bartekpacia and implements a basic assertVisual flow which is much awaited in the community (#1222).

Given a simple React Native app

app.tsx

export default function App() {
  const [toggle, setToggle] = React.useState(false);
  return (
    <View style={styles.container}>
      <View style={[styles.dot, {backgroundColor: toggle ? 'red' : 'blue'}]}>
        <Button title="test" onPress={() => setToggle(!toggle)}></Button>
      </View>
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    alignItems: 'center',
    marginTop: 200,
  },
  dot: {
    backgroundColor: 'red',
    padding: 30,
    borderRadius: '50%',
  },
});

and a flow

test-flow.yml

appId: com.example.app
---
- launchApp
- tapOn: 'test'
- assertVisual: Login screen # compares the current screen against `.maestro/visual_regression/Login screen.png`

maestro will compare screenshots, producing result image with differences masking.

 ║  > Flow: test-flow                                             
 ║                                                                
 ║    ✅   Launch app "com.example.app"                                                              
 ║    ✅   Tap on "test"                                          
 ║    ✅   Tap on "test"                                          
 ║    ❌   Assert visual difference with baseline Login screen (threshold: 95%)                                                     
 ║   
Baseline Current Result
Login screen image Login screen

Result for now is being saved in .maestro/failed_visual_regression/ directory.

If there's no baseline, current will be saved as one.

What is missing:

  • changes accepting flow
  • cloud integration
  • testing
  • remove system-ui elements (like status-bar)
  • component-scoped tests (partial screenshots for component of given test-id)

Testing

  • in progress - test plan has not yet been laid out due to potentially big changes in the proposed flow.

Issues fixed

  • #1222

TheKohan avatar Oct 07 '24 10:10 TheKohan