Tests can fail with kAXErrorServerNotFound

[friedbunny]

Tests on CI can spuriously fail with:

ExamplesUITests
    ✗ testEveryExample, Failed to get matching snapshot: Error getting main window kAXErrorServerNotFound
ExamplesUITests
  testEveryExample, Failed to get matching snapshot: Error getting main window kAXErrorServerNotFound
  ExamplesUITests.m:168
	 Executed 1 test, with 1 failure (0 unexpected) in 270.239 (270.242) seconds

Saw this in two recent PRs: #354 and #356.

[friedbunny]

This seems to manifest in a few different ways, but always with the UI test application dying:

https://app.bitrise.io/build/2d5a7b9eb9084967 from #352

t =   110.00s Add a line style layer from GeoJSON (Swift)
    t =   110.02s     Tap Cell
    t =   110.02s         Wait for com.mapbox.examples to idle
    t =   110.05s         Find the Cell
    t =   110.05s             Requesting snapshot of accessibility hierarchy for app with pid 2829
    t =   110.18s             Find: Descendants matching type Table
    t =   110.18s             Find: Descendants matching type Cell
    t =   110.18s             Find: Element at index 15
    t =   110.19s         Check for interrupting elements affecting Cell
    t =   110.20s         Synthesize event
    t =   110.23s             Scroll element to visible
    t =   110.23s             Find the Cell
    t =   110.24s                 Requesting snapshot of accessibility hierarchy for app with pid 2829
    t =   110.36s                 Find: Descendants matching type Table
    t =   110.36s                 Find: Descendants matching type Cell
    t =   110.36s                 Find: Element at index 15
    t =   110.43s         Wait for com.mapbox.examples to idle
    t =   110.50s     Get all elements bound by index for: Descendants matching type Button
    t =   110.61s         Assertion Failure: ExamplesUITests.m:173: Failed to resolve query: Lost connection to the application (pid 2829).
    t =   110.62s Tear Down
Test Case '-[ExamplesUITests testEveryExample]' failed (110.619 seconds).
Test Suite 'ExamplesUITests' failed at 2019-09-20 16:58:03.350.
	 Executed 1 test, with 1 failure (0 unexpected) in 110.619 (110.620) seconds

[friedbunny]

Not really sure yet what the cause is here, but I suspect it’s Xcode 11 funkiness.

I’m also intrigued by the possibly of moving to individualized tests of examples (#333), rather than the current monolithic testEveryExample — it’s hard to know if one long XCUITest is more or less reliable than many short ones.

[srafiqa]

I am facing the same issue when running from CI system, when running with XCODE 11 and fastlane Error as : , Failed to get matching snapshots: Error getting main window kAXErrorServerNotFound[0m also adb crashed in

[markanderson4]

I'm seeing this error as well, running XCode 11.1

[eb-smith-affirm]

Me too. I'm running Xcode 12.2.1. It seems to be an Apple bug with no work around known but I'd love a solution.

[oliver98844]

Same with Xcode 11.2.1

[iosdeveloperIN]

I am facing same issue on Xcode 11.2.1.

[v-i-s-h-a-l]

Me too. I'm running Xcode 12.2.1. It seems to be an Apple bug with no work around known but I'd love a solution.

Woah, are you from future :P

[VasilijSviridov]

same here xCode 11.2.1 happens 1 in 5 runs :/

[VasilijSviridov]

and basically happens when you using wait for element function, but if you'll give that function some kind of Thread sleep for at least 10 seconds, you'll be alright, not the best workaround but works for me :/

[eb-smith-affirm]

@v-i-s-h-a-l I am from the future but don't tell anybody.

Seems a little better in Xcode 11.3.1 but sure it's fixed yet.

[satheeshkumarf]

Not sure, why we still get this error "Failed to get matching snapshots:", even though the XCUITest documentation says it uses remote queries and there is no snapshot at all… Any insights on this?

[captainbarbosa]

Might be seeing this again with Xcode 12 in https://github.com/mapbox/ios-sdk-examples/pull/411#issuecomment-692339227.

[GintsRomanovskis]

Is there any fixes available?