Provide the option to save failed iOS screenshots to option with flagged name

[MartinMajewski]

Hi there,

we are using the snapshot library on our Azure Pipeline and want to get a failed comparison against an existing reference to be written out as Pipeline Artifact. However, I haven't seen any option to do so. The newly taken screenshot is saved on that simulator device which in azure is not accessible after the job run. Therefore, we have no access to the newly taken screenshot and cannot analyze the differences. With the provided reference images it is no issue, as the worker's path is easily accessible and can be published to the Artifacts store.

Would you mind giving us a hint on how to get the comparison file/report out of the device, please? The assertion messages written to the azure CLI are also less informative than in XCode.

Best wishes, Martin

[MartinMajewski]

Another point:

I have now figured out, that the snapshotting tool provides the reference, failure, and difference picture as attachments at the report navigator window. However, somehow this attachment is not recognized by the Azure pipeline, and therefore it is not published to the test run results.

[pwallrich]

You can use xcparse (https://github.com/ChargePoint/xcparse) to extract the image from the xcresult. I’ve used it to upload the diff as artifacts in gitlab ci if tests failed.

[MartinMajewski]

Would you mind providing me an example of your command for extracting the xcresults, please?

[MartinMajewski]

If someone is also wondering where the xcresults bundle is stored in an Azure Pipeline, please have a look at my comment here:

https://github.com/ChargePoint/xcparse/issues/71#issuecomment-995899727

[luizmb]

Just wanted to share in case someone is using fastlane, just provide the parameter 'result_bundle: true' to the run_tests command (https://docs.fastlane.tools/actions/scan/) and then from a Makefile (or possibly a bash script) you can do something like this:

bundle exec fastlane test && \
		echo "success!" || \
		{ \
			echo "failure!"; \
			mint run ChargePoint/xcparse xcparse attachments test_output/MyApp.xcresult test_output/artifacts --uti public.plain-text public.image; \
			exit 1; \
		}

(Using Mintfile with reference to ChargePoint/[email protected]).