Is there no way to detect which simulator the screenshot came from?

[kumarpatel]

I love this solution sooo much. It's so fast to run making it almost difficult to justify not using it.

It even runs on multiple simulators open at the same time. incredible. it just doesn't have the device names on each of the screenshots.

I'm fairly certain this issue was described in Roadmap issue But thought I'd check in to see if there's any new updates/developments in this space that can make this possible. https://github.com/wix-incubator/rnsst seems to be trying to solve similar problems using Detox @oblador @techeverri

[kumarpatel]

@lpatrun I made some progress, but it's kind of silly and weird.

• I forked React Native Storybook Screenshot Testing
• Modified this line in the file to inject device.takeScreenshot(${kind}-${name}); instead of taking screenshot using detox-applitools-testing
• Then, to include platform and device names in screenshot file path, I used Artifacts pathBuilder property in Detox Configuration for custom path builder logic.
• We have to write our own logic to determine if screenshots should be saved as reference or actuals depending on if reference dir exists.
• By this point I have reference and actual screenshots separated by platform and devices.
• Then I used this awesome suggestion by kirilldronkin to use reg-cli for html report. Only difference here is that I use it for diffing too.

This works as I would expect.

• Opens the simulators I have defined in my detox.config.json one after the other. (or I can run my test against a specific iOS simulator version)
• Installs the app (I added logic in my entry file to open StorybookUI depending on env var)
• Detox runs through all stories and takes screenshots and saves them in the appropriate dir; i.e. reference/ios/iphone-12, current/ios/iphone-12
• Then I manually run above mentioned reg-cli script with minor modification to do the diffing as well.

This, by its very nature, is slower than loki because it has to spin up, install app and spin down the device. It's hands-off and so much faster than any e2e test suit, that I might just end up using this frankenstein's monster version of this setup.

I'm conflicted here because loki, the way it functions right now, is so much faster for my development workflow that I prefer it over anything else. But there's a point in my workflow right before I push something up for review, I want to test on other devices 😭 and I think I would prefer using this, slightly slower, detox solution in this case

[techeverri]

It even runs on multiple simulators open at the same time. incredible. it just doesn't have the device names on each of the screenshots.

I don't really know what you mean by “device names on each of the screenshots” Providing an example of the scenario and the expected behavior helps everyone to understand

In any case, you can do a lot of things to personalize the name of the files generated by Loki Check the official docs on the fileNameFormatter function 👇 🔗 https://loki.js.org/configuration.html#filenameformatter

https://github.com/oblador/loki/blob/8beda16fcda4841c4e23c7353880e94498cbadcc/packages/runner/src/commands/test/get-output-paths.js#L9-L10

By default, it produces files with names similar to the ones in the screenshot below 👇 🔗 https://github.com/oblador/loki/tree/master/examples/react/.loki/reference

[lpatrun]

@techeverri thank you

[kumarpatel]

@techeverri Upon looking into it further, you are correct. It is adding the device names (based on my configuration object) on screenshot files. Here's my scenario and the outcome. Maybe you can help me figure out what exactly is the problem here.

The scenario, (see attached screenshots)

• I have a single Story that displays the device name using react-native-device-info lib
• Open iPhone 12 simulator. Launch app with Storybook initialized.
• Open iPhone SE simulator. Launch app with Storybook initialized.
• Run loki update
• This generates 2 files with correct names. (seems promising)
  • ios_iphone12_DemoScreen_Behaviour.png
  • ios_iphoneSE_DemoScreen_Behaviour.png
• The images captured though are from the same simulator, iPhone 12. Even though both simulators we cycling through stories when loki was running.

The issue seems to be that the screenshot captured isn't actually from the simulator the filename claims it was captured from.

Here's what the simulators actually rendered.