Implement backdoor feature

[codebutler]

Description

• This pull request addresses the issue described here: #207

This makes it possible for tests to send arbitrary messages to the app being tested for the purpose of configuring state or running any other special actions.

Usage inside a test looks like:

await device.backdoor({ action: "do-something" });

Then receive the event in a React Native app:

const emitter = Platform.OS === "ios" ? NativeAppEventEmitter : DeviceEventEmitter;
emitter.addListener("detoxBackdoor", ({ action }) => {
  // do something based on action
});

Inspired by a similar feature in the Xamarin test framework: https://docs.microsoft.com/en-us/appcenter/test-cloud/frameworks/uitest/features/backdoors

---

For features/enhancements:

• [X] I have added/updated the relevant references in the documentation files.

For API changes:

• [X] I have made the necessary changes in the types index file.

[cjshearer]

I was just looking for this functionality a week ago! I'm surprised it's not already a feature, but glad to see I'm not the only one who wants this.

My workaround for this is a hacky mess involving a mock entry file that listens for a launch argument that injects some data into the app. This gets worse, as the mock is only loaded in debug builds, so in release builds the data is not injected, causing the test to fail. Of course, I could bundle this test data in release builds as well, except that would increase the release bundle size with junk data. The only good solution to this is passing in the test data at runtime.

Merging this PR would really help me clean up my code, as I could simply pass data through the backdoor instead of bundling it with a debug build.

[asafkorem]

Hey @codebutler! First of all - thank you for the pull request!

I went through the changes you implemented in this PR, I must say that this is definitely an interesting idea. However, I'm still not convinced that Detox (or generally, E2E testing framework) should have it, can you please provide me with some use cases (or even better, your use case) where you think this feature is necessary?

We'll have to consult about this change internally, probably next week (with @d4vidi, which is OOO this week), and to understand whether it is desirable. This change might also interfere with our current plans and work (reimplementing Detox-iOS with XCUITest framework).

[asafkorem]

@cjshearer can you please elaborate about the kind of data you'd like to pass the app on run-time, and what flows are you trying to test?

[noomorph]

Looks good to me, I'll vote up. The discussion on this PR is ongoing internally.

[cjshearer]

@cjshearer can you please elaborate about the kind of data you'd like to pass the app on run-time, and what flows are you trying to test?

Our app stores data recorded from a proprietary BLE device as JSON in AsyncStorage. These recordings are displayed in a list that the user can select from, allowing them to view a graph of the recording data and some metadata for these recordings. We want these automated tests to run on iOS and Android simulators, so we cannot have it interact with any Bluetooth simulator. We do have plans to mock the BLE device, but we would still like to have a quick way to inject these recordings without actually making them at test-time (some of the recordings we would like to inject would be hours long). Our app supports importing recordings shared to it, so it would be convenient to simply hook into that functionality with the proposed backdoor feature.

Let me know if you need anymore info!

[codebutler]

Hey @codebutler! First of all - thank you for the pull request!

I went through the changes you implemented in this PR, I must say that this is definitely an interesting idea. However, I'm still not convinced that Detox (or generally, E2E testing framework) should have it, can you please provide me with some use cases (or even better, your use case) where you think this feature is necessary?

Thanks for taking a look! My thinking was that this is an appropriate feature for a "gray box" testing framework.

We have "developer" options in our app to tweak internal state for flows that are difficult to test. This makes it easy to hook into that existing functionality.

We'll have to consult about this change internally, probably next week (with @d4vidi, which is OOO this week), and to understand whether it is desirable. This change might also interfere with our current plans and work (reimplementing Detox-iOS with XCUITest framework).

Can I make any changes now to better prepare for that work?

[noomorph]

@codebutler I guess what you also need is to implement some simple handler in the detox/example e2e test app and at least 1 E2E test.

[noomorph]

@d4vidi @jonathanmos I love this feature and I'd like to see it available in some time.

It should serve a good purpose in lightweight integration tests, maybe Storybook-ish stuff, screenshot testing, etc. It might help get rid of some developer-only screens we use for some E2Es. It seems truly useful.

If you approve the idea, I can take charge of this PR and help @codebutler to get it ready for merge.

[d4vidi]

Being a UI testing tool, normally we would advise doing this through a custom test screen with custom mock options. ATM I don't understand what in the the BLE use case makes it special. That said, I would consider that in terms of convenience, although, the API (backdoor) must go out under a different name. Also, I would try to change the impl. such that it would be performed truly synchronously (namely, send backdoorDone message only when handling completes).

I think the right course of action in developing this idea would be to set up a sync discussion on it among ourselves (aka a meeting).

[d4vidi]

BTW @codebutler as we've already merged the unit tests change, we would appreciate you rebasing over origin/master 🙏🏻

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For more information on bots in this reporsitory, read this discussion.

[stale[bot]]

The issue has been closed for inactivity.

[asafkorem]

@wix/detox-team guys, I don't want this PR to stay open forever. Are we going to close it or to discuss on it?

[wouterh-dev]

For what it's worth I would be very interested in this functionality 🙏

It would make implementing custom mocks much easier.

[noomorph]

I still think this PR is awesome, and it is disheartening that we didn't finish it and adopt it to Detox. Maybe I got the wrong impression, but I think it could have made it easier to integrate with Storybook and Kompot.

[noomorph]

My technical concern so far is that it looks like adding asynchronous callbacks is not possible (addEventListener).

A somewhat acceptable workaround would be to "wait until the app is ready", but I'd prefer to verify it wouldn't be flaky due to possible race conditions.

[abajajct]

Would this work allow us to change the date during runtime of test?

[noomorph]

Technically yes, but that would be achievable with an advanced mock server as well (so nothing actually unique to the backdoor feature) - just less legwork, yet still some good old mocking.

[codebutler]

Hi, I'm no longer working with this project so I'm closing this. PR I agree that most (all?) of these uses cases can be handled by a custom mock server instead of this approach. Thanks.

[noomorph]

@codebutler thanks for your hard work. I still have a hope to bring this feature to Detox in one of major updates and I'll give you all the proper credit if/when that happens.

[wouterh-dev]

@noomorph I'm still very interested in this functionality. We are currently accomplishing something like this via device.openURL and https://reactnative.dev/docs/linking#handling-deep-links