FB9649221: Accessibility APIs return -25211 (apiDisabled) when running Xcode UI Tests under Monterey

[godbout]

• Date: 2021-09-24
• Resolution: Open
• Area: XCTest
• OS: macOS 12
• Type: Incorrect/Unexpected Behavior

Description

Please describe the issue: the tests run well on Big Sur. both my app and Xcode Helper are having the Accessibility permissions. under Monterey, the UI Tests fail though. the AX APIs return -25211 which is the code for not having Accessibility enabled.

Please list the steps you took to reproduce the issue:

1. create a basic Xcode project with UI Tests
2. make a function that tries to get the kAXFocusedUIElementAttribute with the AX API
3. the kAXFocusedUIElementAttribute returned is nil and the AXError is -252114.

What did you expect to happen?: as under Big Sur, the AX APIs should work, kAXFocusedUIElementAttribute should be the UI focused element.

What actually happened? kAXFocusedUIElementAttribute is nil. the error is apiDisabled, while the app and Xcode helpers are enabled. the UI Tests fail but the app itself can be launched and works.

[godbout]

Apple Feedback:

Engineering has requested the following information regarding your report:

Could you please provide a sample project that reproduces this along with a result bundle and a sysdiagnose while reproducing the issue?

===========

Important: Note the date and time issue occurred and add this information to Feedback Assistant.

macOS sysdiagnose Instructions:
https://developer.apple.com/services-account/download?path=/OS_X/OS_X_Logs/sysdiagnose_Logging_Instructions.pdf

____________________________________
For a complete list of logging instructions visit:
https://developer.apple.com/bug-reporting/profiles-and-logs/

[godbout]

thanks for the response. i'm currently developing on the latest Big Sur. testing on Big Sur and Monterey. once Monterey is out for the public i'll switch my main development machine to it and will create the most basic app i can to reproduce the issue. thanks.

[godbout]

ok, i've updated my main machine to Monterey and i'm having exactly the same issue. AX returns apiDisabled and all my UI Tests fail. i will give you a sample project and the required diagnostics.

[godbout]

here you go:

1. the project can be downloaded on GitHub at: https://github.com/godbout/Monterey-AX-UITests-apiDisabled
2. i attach the sysdiagnosis and the xcresult files
3. the issue happened between 19:31:11 and 19:31:13

it works fine under Catalina on an iMac and on Big Sur on the same machine.

Files

Test-Monterey-AX-UITests-apiDisabled-2021.10.26_19-31-01-+0800.xcresult.tar.zip

[godbout]

i was able to solve the problem on my local machine by adding manually all the UI Tests Runners to the Accessibility in the Security & Privacy. so i can run the tests locally, but this will fail for sure on various CIs. Xcode and Xcode Helper used to be sufficient to run the UI Tests and call AX functions. now it does not seem so.

[godbout]

Apple Feedback:

Thanks for your update. We have additional request.

Please file a separate feedback report asking for XCTest APIs wrapping the AX usage.

We need to know what AX APIs you are calling.

Please update this feedback report when that has been done, and provide the new feedback ID.  Thank you.

[godbout]

new Feedback: FB9760096.

[godbout]

ok. i don't really get it as in the project that shows the issue (https://github.com/godbout/Monterey-AX-UITests-apiDisabled) it's clear what AX APIs are called: https://github.com/godbout/Monterey-AX-UITests-apiDisabled/blob/master/Monterey-AX-UITests-apiDisabled/AXEngine/AXEngine.swift#L10

but ok, will do.

[godbout]

new Feedback: FB9760096.