Detox
Detox copied to clipboard
wix
iOS: XCUITest integration
[!IMPORTANT]
This is no longer an epic, since we decided to do the integration in smaller steps. Initial XCUITest integration is now part of iOS: Support System Dialogs epic.
Motivation
Currently, there are several gaps in Detox iOS we would like to eliminate, especially the non-support of these two:
- Multiple apps flows (tests that involves multiple applications).
- Real devices (running tests on a physical device).
There are other features we would like to implement and support but we are limited to doing so, infrastructure-wise:
- Web-view interactions.
- Unmocked push notifications flows.
- System dialog interactions.
Currently, in Detox, the entire interaction with the app under test (or, AUT), is done from the JS side, communicating with the Detox framework, in which resides inside the app, aware of all the resources (DetoxSync for instance), and triggers UI events from within. This way, Detox is limited to interact only with the AUT.
Note that, running apps on physical devices, and specifically UI tests, will require code-signing the application with signing certificates issued by Apple.
Also, on iOS Simulators, Detox framework is injected dynamically into the application (using the DYLD_INSERT_LIBRARIES environment variable). For physical devices, inserting dynamic code to the application will require to code-sign the dylib with the same signing certificate of the running application (Apple Code Signing).
Since in many cases our users do not use code-signing on their applications on their local environment, we would like to keep this step optional (unless running on real devices).
XCUITest
XCUITest is a user interface testing framework launched by Apple in 2015, the XCUITest framework rests upon two core technologies: the XCTest framework and Accessibility.
The UI testing in XCUITest is based on the semantic data about the UI that is exposed as "Accessibility", it works by locating the UI elements using queries, sending UI events to these objects and providing a rich API enabling us to examine the elements' attributes and state, and comparing them against an expected state.
The transition to accessibility-based operations, will allow Detox to interact with multiple apps, iOS SpringBoard, push notifications, system dialogs and other system elements all managed from the test target process, without having to inject any dynamic code injected into the AUT. We should also be able to operate on apps without the additional phase of inserting dynamic code.
This is an amazing summary. Very well articulated. I consider this the new feature Epic, then.
how we looking on this? I saw we now have android web view support. so close to bailing on appium...
Are you guys planning to use EarlGrey 2 for achieving this? This is such an important feature. There's a huge enterprise opportunity with the adoption of Ionic. Being able to interact with a Webview would be key for Ionic apps :)
Are you guys planning to use EarlGrey 2 for achieving this? This is such an important feature. There's a huge enterprise opportunity with the adoption of Ionic. Being able to interact with a Webview would be key for Ionic apps :)
No, we don't have any plans to use EarlGrey 2 for that. I did not get to check ionic at all, this sounds very interesting. Thanks :+1:
our app's login module is inside a web view which works for android but i'm eagerly waiting for detox team to release the same support in webview for ios as well
@asafkorem Shall you need any help for testing the feature on an Ionic app (webview based) just ping me! I'm really looking forward to introduce Detox as our official E2E testing framework for mobile apps ;)
Sounds good, thanks @guzmanoj, that would be great if we could use Detox on ionic apps :+1: I guess we'll have to do some extra adaptations for ionic apps, but we should definitely try this.
Hi!
What's the progress on this? I see most of the Phase 1 issues are closed but no PRs ale linked.
It's actually in the works. Is there anything specific you're planning on doing once it is done? Current ETA is Jan 23
It's actually in the works.
Great! Is it open-source?
Is there anything specific you're planning on doing once it is done?
I'm developing a new UI testing framework for Flutter and was wondering how you guys solved the problems mentioned in this issue :) I'm especially interested in making XCUITest functions, such as tapping and entering text, work inside XCTest.
I'm getting the following errors when initializing UIApplication()
*** Assertion failure in -[XCUIApplication commonInitWithApplicationSpecifier:device:], XCUIApplication.m:187***
Device is not configured for UI testing - use of XCUIApplication is not supported. This can happen when XCUIApplication is used in a unit test bundle instead of a UI test bundle. (NSInternalInconsistencyException)
Yes @bartekpacia, and I'd be happy to share my knowledge and learn more about your work on this project. As far as I know, Flutter doesn't use the standard UI hierarchy and elements of native platforms, so this might be a problem to achieve this with XCUITest, but please correct me if I'm mistaken. If you're interested, we can continue this conversation on Discord, my username is Asaf Korem#7718. 🙂
Hi @asafkorem, thanks for your will to help :) Fortunately, we've already managed to implement this feature. See https://github.com/leancodepl/patrol/issues/661 for more.
looking forward to seeing new features and implementation to support about mentioned limitations. it's amazing to support and believe that most of us face these issues. I'm willing to join test once it release. hope Detox become the best UI testing framework for cross-platform and native apps
looking forward to seeing new features and implementation to support about mentioned limitations. it's amazing to support and believe that most of us face these issues. I'm willing to join test once it release. hope Detox become the best UI testing framework for cross-platform and native apps
Thanks @aaronpliu. You are more than welcome to start testing out our XCUITest (i.e. Detox v21) RC, today!
Are react native views also accessed with XCUITests in the new version? There is a limitation of XCUI Test to only be able to access nested elements with a depth of 60. this is an open issue of appium for a very long time. https://github.com/appium/appium/issues/14825 i hope you are aware of this issues
I'm also interested to know if Detox can handle nested elements that have a depth of more than 60 (in regards to @chrisgradl's comment), as this has been an issue for a very long time in Appium and we are currently unable to test our iOS app due to this issue with Appium.
Considering making the swap to Detox but I'd like to know if we will run into this same issue using Detox.
@chrisgradl @matiaswastaken we were not aware of this issue but that's not to say we'd be affected by it. We will check (@asafkorem wdyt?). Do you have a use case where you could give the Detox 21 RC a try and see whether it breaks? In any case, I do wonder what app has views nested all the way down to a depth of 60, tbh. I think Apple might have been trying to blatantly hint that the right course of action when tests fail is to stop and revisit the app's UI.
@d4vidi If I remember correctly, the common cause of this is using the React Navigation UI libraries, which apparently use a lot of nested elements.
I'm not sure why the deep nesting is necessary, but it is a popular library so there are probably a lot of Detox users that also use React Navigation.
This bug is actually the main reason I switched from Appium to Detox. I would guess that this change may break tests for a lot of people.
@d4vidi Hi, The issue is reported in here with sample application code to reproduce it. I'm also interested in if the issue is reproduced with Detox or not.
Interesting case, I wasn't aware of it. Thanks for bringing this to our attention. We will definitely try to come with an investigation and solution from our side in case this will affect Detox tests as well.
@d4vidi Hi, The issue is reported in here with sample application code to reproduce it. I'm also interested in if the issue is reproduced with Detox or not.
Thanks. We will look into it once the upcoming release of Detox 21 fully materializes 🙏🏻
Hi, looking forward to this. Is there a new ETA for the Detox 21 release?
For those waiting for the release of this version specifically for WebView testing capabilities, it's important to highlight that WebView-testing APIs are already available on both platforms, independent of XCUITest.
Comprehensive information can be found in the documentation: API, guide.
This is no longer an epic, since we decided to do the integration in smaller steps. Initial XCUITest integration is now part of iOS: Support System Dialogs epic.