Detox
Detox copied to clipboard
wix
Run on hardware iOS device
There are several challenges we need to overcome before we can achieve this. Please know there there will be some limitations to what will be possible on the device, such as permission changing.
Here is how we achieved this recently.
- We took the "Copy Detox Framework" run script from the demo-react-native example project.
- As I ran into code signing issues, I replaced it by manually embedding the Detox.framework into the project.
- I added a new script that recursively code signs each framework and dylib inside Detox.framework.
- I copied and embedded the XCTest.framework also in the project as Earlgrey was complaining about it.
- I set up the detoxServer and detoxSessionId in the launch arguments of the scheme.
- The device and laptop was on the same network of course. The laptop's IP was set up correctly in the appdelegate and in the detoxServer launch argument.
- I set up the same port and sessionId in the detox config (in package.json)
- Also added an "ios.none" config there.
- I ran "detox run-server" manually
- Then, I ran "detox test --configuration ios.none".
And now, about the problems. After setting these all up, the app was running, it connected to the server, the tests started to run. And each of them failed. :|
During investigating what could be wrong, it suddenly started to work. Without any code change. The same pattern repeated on one of my colleague's machine. (It failed the tests first, then it passed them half an hour later.)
@simonracz did you run a react-native app? in debug mode ? Loading the bundle from the debug node server may take time, and detox may reach the default timeout and fail.
If you mean the Mocha runner time out, then no, it wasn't that.
We tried both release and debug mode. It was the demo-react-native sample. No modifications were made to it, apart from the project file modifications mentioned above.
The first test line is "await expect(element(by.id('welcome'))).toBeVisible();". This failed with an Earlgrey assertion. It found the view with the id welcome, however it failed the visibility test.
I started to debug the runtime view hierarchy. For example, I stopped the process and called "po [GREYElementHierarchy hierarchyStringForAllUIWindows]".
For this view it showed the AX.id correctly, but the AX.frame was (0,0),(0,0). Sadly, I don't have saved logs.
After it started to work, the AX.frame was correct.
I wasn't able to reproduce it since then.
I was able to reproduce this "Earlgrey fails the visibility test" issue.
In fact I spent half-a-day on chasing this. I was able to narrow it down to the following.
The matched view is rejected, because it's accessibilityFrame is zero. It's frame is correct. It is an RCTView, which inherits from UIView. From to the docs "The default value for this property is CGRectZero unless the receiver is a UIView object or is a subclass of UIView, in which case the value is the frame of the view."
So, this shouldn't happen unless someone manually touches the accessibilityFrame. I looked at the react-native source-code, they never touch the accessibilityFrame. Neither does Earlgrey (except in it's tests).
I restarted, cleared, rebuilt, killed everything. Nothing helped.
I don't think this is a Detox issue. I wrote it down only that in case you encounter it, let's share our findings.
After spending more than a day on this bug I found the issue.
Since some iOS version (don't know yet exactly when, but 10.3.2 and 10.3.3 are affected) Apple introduced a bug/feature around accessibility.
The accessibilityFrame is not set, nothing isAccessible etc. unless the user is really using some accessibility features, e.g. voiceover.
Earlgrey heavily depends on accessibility. I don't know why haven't anyone else found this, opened a bug about it yet.
I'll open a bug report on Earlgrey and also open radar. I want to determine first the exact iOS version when this was introduced.
@simonracz Interesting. XCUITest also uses accessibility for its operation. It’s worth investigating how the system allows that for those cases but doesn’t for ours. Most likely an env var.
@simonracz This is probably an intentional change. There is no need to calculate these values on a real device, running production software (I don't mean a release build). We need to find out how the system differentiates between running from Xcode and otherwise. I will take a look soon.
@LeoNatan is it possible to run on real device now ? I couldn't find any docs on it
@LeoNatan This will be so helpful to have! Thanks for putting in the work. Can't wait to get my hands on it.
@pachun Please do not unassign members of the team from issues. Leave the issue management to us. Thanks.
@LeoNatan I have no idea how that happened. Sincerest apology. 100% unintentional. All I meant to do was thank you.
@LeoNatan, are there any updates on the iOS real devices support? We are really looking forward to this functionality...
Nothing to announce yet. It’s in our backlog and we intend to investigate this eventually, but it’s not on the top of our priorities. There are some challenges that we haven’t solved yet.
Really looking forward of this too. My app need turn on the real device to check the camera.
@LeoNatan Don't you think the following note in README should be adapted then?
Runs on Devices: Gain confidence to ship by testing your app on a device/simulator just like a real user.
IMO, it is a bit misleading. I would mention it is not yet supported on iOS.
@DanielMSchmidt Sure I can!
Do you know if running on physical device on Android works? I have seen android.attached config in the docs but I didn't test it so I'm not sure.
I'll adapt the README accordingly.
AFAIK it should
Fidan Hakaj [email protected] schrieb am Mo., 8. Jan. 2018 um 15:49 Uhr:
@DanielMSchmidt https://github.com/danielmschmidt Sure I can!
Do you know if running on physical device on Android works? I have seen android.attached config in the docs https://github.com/wix/detox/blob/master/docs/Introduction.Android.md#4-add-android-configuration but I didn't test it so I'm not sure.
I'll adapt the README accordingly.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wix/detox/issues/95#issuecomment-355986544, or mute the thread https://github.com/notifications/unsubscribe-auth/ABRm1t2L4xtCOM1DmKekI5Qa5isrXHd0ks5tIisBgaJpZM4MVY2O .
I think I've solved the first hurdle (but a big one), which is setting environment variables when running on device. 🎉
Using a modified ios-deploy, I was able to embed environment variables.
Steps ahead: sign Detox dylib with the same signature as the user's app and send to device storage, then try to inject it using DYLD_INSERT_LIBRARIES environment variable.
Could anyone please walk me through how signing is managed on test clouds?
No new progress. We've done a POC and it's technically feasible. An actual implementation is planned, but there is no timeline.
@LeoNatan since it can run on a real device will it be possible to somehow integrate Detox tests into firebase test lab? Especially now with iOS support announced, could be very powerful.
