[Epic] Add fully-blown support for Genycloud emulators

[d4vidi]

A follow-up on the work on Genymotion-cloud integration done in the context of a grand PoC in #2429. While #2429 sums up the work required in order to start rolling out Genymotion cloud support internally at Wix, there is more to be done in case the integration proves scalable and robust - so as to make it a polished, generally used solution provided via Detox. Here is the list:

• [x] Better tweaking of Detox' CI script, so as to support exec on both local and cloud emulators in each build. (#2688)

• [x] USER_AGENT=detox for any usage of detox out there (as an overridable default) (#2691)

• [x] Fix: Avoid retries for instance creation, as retries can yield an "instance name already exist"-style errors, and ultimately shadow the real, primal error. (#2763)

• [ ] Fix bug: ios runs after genymotion -> teardown is called for all instances, and fails. Need to delete clearing db file in each run.

• [ ] Reduce instance readiness polling: delay retries for ~50 sec (i.e. reduce load on Genycloud's servers and log clutterring)

• [ ] Documentation + scaffolding script revisit. Important: must cover the global Detox init and teardown. Important 2: Must cover mocha.js.

• [ ] [wip] Revisit some technical debts (PR feedback notes, possible filenames change, internal TODO) [#2573, #2575, #2577]

• [ ] Dependency Injection - InversifyJS / di4js

• [x] [tbd] Allow for running debug mode w/ or w/o some sort of instance reuse

• [ ] [tbd] Add support for running on a specific cloud instance (mostly for local dev usage / debugging)

• [ ] Optimize post-testing idle usage time by introducing a worker-teardown hook into Jest, then use it to kill the associated instance (note: currently, all workers -- and thus cloud instances are kept alive until the very end of the suite).

• [ ] Blogpost? :)

[nhoughto]

Would be great to support a way of hooking into geny motion device naming too, we want to create a fallback device cleanup process to cleanup devices whose CI job has completed, but the Detox-<timestamp>-<jest index> can't be related back to a CI job, so there is no way of knowing which device is for which job.

As we cant guarantee that the detox device cleanup process will always fire a fallback is required.

[d4vidi]

Hey @nhoughto, quite exciting to see interest in this feature so soon (hasn't yet officially launched...) Detox does take care of cleaning up all used Genymotion-cloud instances. You have to issue a call to detox.globalInit() in Jest's global init script, and an equivalent detox.globalCleanup() call in Jest's global teardown.

When all of this is mature enough, we will properly document it (as mentioned in the issue' description 😄)

[nhoughto]

Yeah I figured it wasn’t quite finished yet, still works quite well.

globalCleanup is useful, but can’t solve the entire problem as depending on what CI system you use you can’t ensure the cleanup is actually fired, the job may fail, process run out of memory etc and never run.. would be useful be able to name the device so a cleanup task could relate device to ci job. 👍🏼

[d4vidi]

Admittedly, we are facing the same issues in our use cases. For a more elegant solution, I've personally approached Genymotion with a request to allow for associating custom data with each instance (e.g. build#, url, or whatever). Hopefully, they will introduce it, eventually.

[nhoughto]

Yeah adding tags to the devices would be better, failing that jamming context into the only place we have would work too =) On 10 May 2021, 9:48 PM +1000, d4vidi @.***>, wrote:

Admittedly, we are facing the same issues in our use cases. For a more elegant solution, I've personally approached Genymotion with a request to allow for associating custom data with each instance (e.g. build#, url, or whatever). Hopefully, they will introduce it, eventually. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[d4vidi]

I understand of course :)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this reporsitory, read this discussion.

[kg-currenxie]

Hi. Happy to see support for this, as we're struggling with stability running Android in CircleCI (weird out-of-memory issues, even tho there should be enough)

Just tested Genycloud and to my surprise it works incredibly well! 5/5 re-runs passed, and a bit faster than our current runs. Though I tested with the cli locally, and not in CI, but it looks like it would work the same.

PS: I've tested both with --workers 1 and 2. Both working 👌

---

My only question is, since the issue is still [Open], and my testing was successful, what is left to do here? Could we expect some stuff not working? 🤔

[noomorph]

Hi @kg-currenxie, thanks for the cheering feedback. As for your question:

1. First of all, we will release the official Genymotion SaaS support as a part of Detox 20 in the upcoming days.
2. There are minor issues in some advanced scenarios, e.g., #3573, and #3655. These are rather minor inconveniences, but we're not satisfied with their existence, of course, and we are committed to fixing them within a month or two at most.
3. Last but not least, we want to improve init performance dramatically for projects with numerous test suites by dropping unnecessary app re-installation where possible (this is not Genymotion SaaS-specific, but this is exactly an entry point to fix the aforementioned minor issues).

[kg-currenxie]

@noomorph sounds awesome :D Then we're happy to make the switch <3

[noomorph]

The next (20) version and its docs are basically ready, so you can try to install [email protected] and follow the Migration Guide and Genymotion SaaS guide, to prepare your project in a Git branch towards the upcoming release. I'd be happy to hear feedback if the Migration Guide is missing something or is inaccurate somewhere.

[kg-currenxie]

Just to confirm; does it still support recording videos? :) Didn't see any mention of it in the caveats section of the migration guide :)

[noomorph]

Just to confirm; does it still support recording videos? :) Didn't see any mention of it in the caveats section of the migration guide :)

Is there any specific issue you have spotted?

Well, there are no video-specific changes in Detox 20. The support remains basic - 1 video per test - like it always has been since Detox 8. Advanced video cutting is planned but without an ETA at the moment.

[kg-currenxie]

No, no issues, just didn't have time to test video recording until my free Genycloud minutes ran out 😅 Thanks

[kg-currenxie]

I'll report back when we have done the v20 prerelease migration (next week)

[kg-currenxie]

So far, fairly good! After the upgrades (now v20.0.1), I've tried running it 5 times in a row so far. Same specs, same config, same command, right after each other. (2 workers) Results in seconds:

• ✅ 432s
• ✅ 439s
• ✅ 479s
• ✅ 516s (17% slower than fastest run, but still fast imo)
• ❌ One failed run

It's not quite clear what the failure is. Some kind of network timeout I think. I'll report here, but it might be more on Geny's side. I'll give their support this issue.

Log (trace level, 65k lines): https://gist.githubusercontent.com/kg-currenxie/8e1f428138920489c7afcb5c1096c820/raw/5dfdcfa1d0fb3ab0236c4af2e43cbe61ef4ebacc/gistfile1.txt

Please cmd+f for failed in it :)

[d4vidi]

@kg-currenxie thanks, but it's hard to tell from the log. Would be great if you could share your detox.trace.json artifact (https://wix.github.io/Detox/docs/api/logger/#detoxtracejson)

[kg-currenxie]

@d4vidi i'll provide the json trace next time it fails :) i didnt have --record-logs on :/

For now, i'll give you guys the ones that have big time differences:

399 seconds: detox.trace.json.zip

513 seconds: detox.trace.json.zip

28% time difference, altho, still fast enough for our liking :) would just be interesting to gain any insights on why

[kg-currenxie]

Update: In the slow runs (those ~500s), I found that one spec is much slower, while the rest are quite similar: I narrowed it down to one ✓ Balances should be able to make a transfer from any given balance_ While the spec says that it passed, its log has an error:

11-21 05:33:39.568  2491  2580 I DetoxRNExt: Reloading React Native
11-21 05:33:39.580  2491  2839 I ReactNative: [GESTURE HANDLER] Tearing down gesture handler registered for root view com.facebook.react.ReactRootView{171f7b V.E...... ......ID 0,0-1080,2028}
11-21 05:33:39.581  2491  2839 E unknown:ReactContextBaseJavaModule: Unhandled SoftException
11-21 05:33:39.581  2491  2839 E unknown:ReactContextBaseJavaModule: java.lang.RuntimeException: Catalyst Instance has already disappeared: requested by DeviceInfo
11-21 05:33:39.581  2491  2839 E unknown:ReactContextBaseJavaModule: 	at com.facebook.react.bridge.ReactContextBaseJavaModule.getReactApplicationContextIfActiveOrWarn(ReactContextBaseJavaModule.java:66)

Which is exactly the same error as I just reported here https://github.com/wix/Detox/issues/3722

Refrence screenshots: Fast:

Slow (the one with an error)

It says that the tap is slow (but i think its a bit misleading maybe), but it seems that it hangs for a bit, and later resumes.

[d4vidi]

Hey @kg-currenxie! This specific problem doesn't seem to be related to Genymotion, so let's follow up on it in #3722 in order to keep things clean over there 🙏🏻