patrol icon indicating copy to clipboard operation
patrol copied to clipboard

iOS Simulator e2e tests are extremely flaky on GH Actions

Open piotruela opened this issue 6 months ago • 5 comments

The tests that we run on the iOS Simulator in the GitHub workflow keep failing. The main cause seems to be this problem: iOS Simulators no longer launching. When I try to run tests on iOS Simulator with Xcode > 15.0 or/and iOS SDK > 17.0 the process is so slow that it easily exceeds timeout duration (and also many tests fails randomly). I tried to use previous versions of macOS/Xcode/iOS and so far the best configuration was [macOS-14, Xcode 14.3.1, iOS 16.4]. By 'the best' I mean that it works quite fast but still, many tests fail randomly without a clear reason. When running the same tests locally everything work as expected.

piotruela avatar Aug 06 '24 06:08 piotruela