[WiP] feat: add device.backdoor() API

[noomorph]

🚨 Blockers🚨 (why it is WiP)

The underlying mechanisms seem to have changed, and this PR does not work already with RN73. 😢

Description

This is a revived pull request by @codebutler (#3245). All props go to him. 🏆

Inspired by functionality in the Xamarin test framework, this PR proposes a new API within Detox to allow tests to send messages or commands directly to the app, modifying its state during runtime without UI interactions.

The "Detox as a platform" vision discussed in issue #207 underlines the potential benefits and use cases of enabling dynamic behaviours and state manipulations during test executions. The Backdoor API is a step in this direction, providing a mechanism for tests to instruct internal changes within the app from E2E tests dynamically.

Implementation:

A simple API call within a test sends a specific action to the app. The message should be an object with action property and any other custom data:

await device.backdoor({
  action: '...',
  /* ... optional serializable payload ... */
});

On the application side, you have to implement a handler for the backdoor message, e.g.:

import { detoxBackdoor } from 'detox/react-native'; // to be used only from React Native

detoxBackdoor.registerActionHandler('my-testing-action', ({ arg1, arg2 }) => {
  // ...
});

// There can be only one handler per action, so you're advised to remove it when it's no longer needed
detoxBackdoor.clearActionHandler('my-testing-action');

// You can supress errors about overwriting existing handlers
detoxBackdoor.strict = false;

// If you want to have multiple listeners for the same action, you can use `registerActionListener` instead
const listener = ({ arg1, arg2 }) => { /* Note that you can't return a value from a listener */ };
detoxBackdoor.addActionListener('my-testing-action', listener);
// You can remove a listener in a similar way
detoxBackdoor.removeActionListener('my-testing-action', listener);

// You can set a global handler for all actions without a handler and listeners
detoxBackdoor.onUnhandledAction = ({ action, ...params }) => {
  // By default, it throws an error or logs a warning (in non-strict mode)
};

Use Case Example

Using the Backdoor API, we can mock time from a Detox test, assuming the app takes the current time indirectly from a time service which has a mocked counterpart (see the guide), e.g.:

• src/services/TimeService.ts
• src/services/TimeService.e2e.ts

From a Detox test, this may look like:

await device.backdoor({
  action: "set-mock-time",
  time: 1672531199000,
});

Assuming the TimeService provides the current time via now() method, the mocked TimeService.e2e.ts will be listening to set-mock-time actions and return the received time value in its now() implementation:

// TimeService.e2e.ts

import { detoxBackdoor } from 'detox/react-native';

export class FakeTimeService {
  #now = Date.now();

  constructor() {
    detoxBackdoor.registerActionListener('set-mock-time', ({ time }) => this.setNow(time));
  }

  // If you have a single instance through the app, you might not need any cleanup.
  dispose() {
    detoxBackdoor.clearActionListener('set-mock-time');
  }

  setNow(time) {
    this.#now = time;
  }

  now() {
    return this.#now;
  }
}

This allows any test to dynamically set the time to a desired value, thereby controlling the app behaviour for precise testing scenarios without multiple compiled app variants or complex mock server setups.

If you still would like to have multiple listeners for the same action, you can use the detoxBackdoor.addActionListener method instead:

const listener = ({ time }) => console.log(`Received time: ${time}`);
detoxBackdoor.addActionListener('set-mock-time', listener);
detoxBackdoor.removeActionListener('set-mock-time', listener);

The weaker side of action listeners is that they will never be able to return a value to the caller by design.

Further development

While this PR introduces the ability to send instructions from tests to the app, it lacks bi-directional communication, enabling the app to report state or responses back to the executing test. This can be a future development vector for this feature.

Here's a draft version of the future BackdoorEmitter (omitting listeners and non-relevant code):

export class BackdoorEmitter {
// ...

  /** @private */
  _onBackdoorEvent = ({ id, event }) => {
    const handler = this._handlers[event.action] || this.onUnhandledAction;
    const promise = invokeMaybeAsync(handler, event);

    promise.then((result) => {
      this._emitter.emit('detoxBackdoorResult', { id, result });
    }, (error) => {
      this._emitter.emit('detoxBackdoorResult', { id, error: serializeError(error) });
    });
  };
}

As you can see, the missing part to finish it on the native side is to tap on the detoxBackdoorResult events and wait for them before returning the result to the Detox tester (client).

Summary

This PR facilitates dynamic mocking and state configuration during Detox test runs, enhancing testing capability and flexibility. Feedback, especially regarding practical use cases and potential improvements, is welcomed.

It can give a simple enough solution for many support requests like in #4088, #4207, #3986 and others.

For features/enhancements:

• [x] I have added/updated the relevant references in the documentation files.

For API changes:

• [x] I have made the necessary changes in the types index file.

[noomorph]

Okay, so far, I have studied the existing concerns from the team about the previous attempt to introduce Backdoor API.

The following points have been raised:

1. Backdoor API naming is undesirable. (@d4vidi)

On the contrary, this is a very accurate name because it reminds the user about the danger of leaving any detoxBackdoor listeners in the production code. See the danger admonition from the updated Mocking guide:

Avoid using detoxBackdoor in your production code, as it might expose a security vulnerability.

Leave these action handlers to mock files only, and make sure they are excluded from the public release builds. Backdoor API is a testing tool, and it should be isolated to test environments only.

Let's go to another point.

2. Backdoor API (at this implementation stage) is not bidirectional – i.e. you can't send the results back from the app (including "I am done" confirmation). (both @d4vidi and @asafkorem)

This makes sense to me – such functionality would be valuable. On the other hand, I don't view this as a critical flaw but as a minor shortcoming acceptable for Phase 1. Since any consequent activity like re-rendering, network requests, timers, etc., falls under the general synchronization handling in Detox, I doubt that we'll see any new idling timeouts due to the backdoor mechanism.

What I did is that I limited the action handlers to "single per action". This limitation is enough to be forward-compatible with Phase 2. For users who need broadcasts (one-to-many), there are action listeners to add and remove.

[markhomoki]

This is a really good idea, and will simplify certain flows.

We are using a similar flow, but we also expect data from the application and use it throughout the test. Application I'm working on is white label, and tests are running on multiple prod environments that have different config. While some parts can use mock files, I'm wondering if this backdoor action could also help us get data back into Detox.

I see this PR sends data to the RN app, but could it be modified to receive data as well? Thanks, and great work. 🚀

[noomorph]

UPD: already addressed in the new commits. Disregard this message (or read it for history reasons).

Well, if there is a way to subscribe to RN emitter the same way JS does, then certainly that is possible.

After a couple of thoughts, the weakest side of this pull request is that this API is not encapsulated:

import {DeviceEventEmitter, NativeAppEventEmitter, Platform} from 'react-native';

const RNEmitter = Platform.OS === "ios" ? NativeAppEventEmitter : DeviceEventEmitter;

RNEmitter.addListener("detoxBackdoor", ({ action, time }) => {
  // Handle actions as needed
});

If it turns out that we can listen to these events on the native side, then it may transform into a pair of detoxBackdoor/detoxBackdoorResult events with some event ids for tracking, and that's why releasing this feature as-is (the way the original author intended), is not the best idea ever.

Maybe a good compromise for now would be to create a facade for this API that end users can import as a part of Detox, and make sure it can be upgraded to bidirectional flow without breaking compatibility. 🤔

[noomorph]

I'll take it for the next sprint – hopefully I'll be able to set up https://github.com/wix-incubator/wml with our e2e project.

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For more information on bots in this repository, read this discussion.

[noomorph]

It seems there is a simpler solution nowadays to this:

https://metrobundler.dev/docs/configuration/#unstable_enablesymlinks-experimental

[noomorph]

I managed to revive the Metro bundler, but @d4vidi @asafkorem it looks like this call is outdated and won't work with the new React Native:

+ (void)emitBackdoorEvent:(id)data
{
	if([DTXReactNativeSupport hasReactNative] == NO)
	{
		return;
	}

	id<RN_RCTBridge> bridge = [NSClassFromString(@"RCTBridge") valueForKey:@"currentBridge"];
	[bridge enqueueJSCall:@"RCTNativeAppEventEmitter" method:@"emit" args:@[@"detoxBackdoor", data] completion:nil];
}

Maybe we need to find direct way to call emit in the emitter, without the bridge. 🤔

I have to stop again here.

[renatop7]

Hello @noomorph, this solution is very cool, do you know if this works in RN 0.71?

I'm desperate for something like this, to be able to control my testing without a bunch of flags inside my app.

[noomorph]

@renatop7 , just as we are speaking, my colleague @andrey-wix is working on this pull request right now. I hope you will see it soon.

P. S. As for RN71, yes, it worked, but I don't recommend to recompile/patch unless you are very proficient in these things.

[renatop7]

@noomorph that's awesome! I'll wait then, hope to see it soon as well :) Thanks!

[stale[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For more information on bots in this repository, read this discussion.