JS: migrate to jest globalSetup, globalTeardown configuration

[d4vidi]

Description

Implement the new entity, Detox root context:

• it should be created once, by using the resolved config (based on environment variables, CLI arguments, and .detoxrc files);
• it should manage the entire test session, running the test runner (jest) as many times as needed;
• it should be able to communicate with the workers in both directions including broadcast.

Remove old hacks we applied in absence of the centralized communication between the workers:

• having many per-process logs (detox_<pid>.log) instead of a single one that combines all the messages;
• adding auxiliary various environment variables (DETOX_REPORT_SPECS, DETOX_START_TIMESTAMP ) to the spawned test runner to make up for the absence of a shared global state;
• demanding to know the workers' count in advance – no need anymore, because the root context will be registering the new workers as they appear.

Introduce the features that immediately make sense upon the refactor:

• logger config – since Detox will be possessing only one global instance of the logger, we can easily enough create one using the config, and all the workers will be using inter-process communication to forward the messages to that instance. That simplifies artifacts' collection and human inspection a lot.
• testRunner config – after removing knowledge about workers, Detox CLI becomes truly runner-agnostic – it can act as a simple passthrough mechanism. Now there's a lot of sense to add a universal testRunner config which can be useful for any test runner.
• *.jsonl to Chrome trace format by default and be able to visualize them nicely, since all the PIDs are in the same place.

In the long run:

• the root context will be used to allocate the devices and keep them in a pool – that should speed enormously the workers because we'll be omitting extra deleteApp/installApp calls;
• we'll be able to fix the --cleanup flag behavior.

Scope

• [x] locate stateful modules and singletons and assign them to the corresponding realms (see below);

• create IPC communication between top realm and the underlying ones so to:
  • [x] reduce Detox config resolution and creation to one time, in the top realm only;
  • [x] dynamically register workers and query their exact count - makes us independent from test runner
  • [x] implement session mechanism to communicate better between primary and secondary contexts
  • [x] restore reportSpecs logic from CLI – get workers count via IPC and set the default value accordingly read properties of undefined (reading 'jest')` - think how to initialize secondary context in a guaranteed way;
  • [x] share Detox configuration via serializing it into a temporary file and providing process.env.DETOX_CONFIG_SNAPSHOT_PATH to the child environments
  • [ ] add [permanent] boolean option to be able to report failed tests that should not be retried
  • [ ] [backlog] add Windows 7/8 compatibility
• simplify Detox CLI+config mechanism:
  • [x] remove knowledge about the count of workers; the option -w, --maxWorkers still remains as a passthrough;
  • [x] remove knowledge about Jest configs; the option -c, --config still remains as passthrough;
  • [x] introduce a new testRunner config section where the user can add extra CLI arguments to pass by default to the test runner via Detox CLI;
  • [x] restore keepLockFile and global lifecycle handers logic from CLI
  • [x] implement Genymotion fallback in global setup and teardown using device;
  • [x] fix broken spawn mechanism (not supporting globals and escaped special characters)
  • [x] make testRunner section extendable by other third-party users (less hardcoded Object.assign's, more _.merge, please)
  • [ ] add compatibility layer for the old runnerConfig/testRunner/runner-config/test-runner strings with deprecation warnings
• Jest-specific features:
  • [x] restore platformSpecificString (--testNamePattern) logic from CLI (via implementing it in test environment)
• rework the user-facing part of Detox' facade:
  • [x] restore compatibility with Detox V19 facade in require('detox')
  • [x] add new Jest entry points: globalSetup, globalTeardown;
  • [x] remove MissingDetox double class which mitigated jest-jasmine2 test execution quirks;
• logging subsystem:
  • [x] merge sorted JSON logs into one artifact log via knowing from the session state the file locations
  • [x] create a bunyan-debug-stream-processed .log version aside the merged JSONL
  • [x] implement logger config section (with loglevel and various bunyan options):
    • level: enum;
    • overrideConsole: boolean;
    • bunyanOptions: object;
  • [x] restore trace and traceCall: implement it back via logging mechanism;
  • [ ] port lifecycle metadata additions by @asafkorem
• bugfixes:
  • [x] move away from the workaround with hot switching stdout/stderr when changing bunyan debug stream options
  • [x] investigate error: CI=1 npx detox test -c stub e2e/01.sanity.test.js e2e/02.matchers.test.js -w 2 --jest-report-specs=false: DetoxInternalError: the tester is already connected to the session 4f6bdfd1-c6a2-7383-0451-e54d4b4e719a
  • [x] investigate error: npx detox test -c stub e2e/01.sanity.test.js e2e/02.matchers.test.js -w 2: `TypeError: Cannot
  • [x] dynamically calculate workers count on CI
  • [x] protect primary context from re-initializing
  • [x] make sure temporary files do not leak
  • [x] fix EventEmitter leakage
  • [ ] fix metadata not showing up in tracing events
  • [ ] rename RuntimeConfig properties to regular DetoxConfig properties no avoid extra confusion
  • [ ] update typing tests
• unit tests:
  • [ ] IPC server and client
  • [ ] session state serialization
  • [ ] JSONL streams manipulations
  • [ ] BunyanLogger
  • [ ] Detox contexts: primary and secondary
  • [ ] fix --inspect-brk test CLI unit test
• integration tests:
  • [x] check the new version in all the possible contexts (regular run, debug run):
    • single process: primary context (+ worker) – simulating old scenario with Mocha
    • single process: primary context and secondary context + worker – simulating jest --runInBand without detox test CLI
    • ~same as above, no local-cli, but with Jest config where Detox init happens (!)~ nope, we use resolveConfig now.
    • parent process with primary context, and child process with secondary context
    • parent - child - grandchildren – simulating multiple workers in Jest via Detox CLI
    • test other local-cli scripts
• docs:
  • [ ] review CLI reference
  • [ ] behavior config addition: keepLockFile)
  • [ ] new logger config section
  • [ ] new testRunner config section

Realms

A realm object abstracts the notion of a distinct global environment, with its own global object. For each stateful module and singleton in our codebase, now it is high time to make it crystally clear in which realm[^1] it runs, because we historally had, still have and will have quite a few realms, 🤯 :

1. root realm: usually Detox CLI, where yargs run, but see the caveats section below. It is the first and the only place which receives all the possible means (environment variables, CLI arguments) to synthesize a working Detox config to use throughout the whole test session. It controls the test runner process and might spawn it a few times if the retry mechanism is enabled.
2. runner realm: this is the global context of the test runner which is responsible for spawning and controlling the test workers, reporting the tests' status after the entire session. In the case of Jest, it runs in a separate process.
3. worker realm: this is likely to be running as the bottom-most OS process, but in the case of Jest, this is not the lowest tier yet – its goal is to create sandboxes for every next test suite file, each of which is running inside a special test environment class.
4. sandbox realm: this is where the user test itself is executed, the most short-living realm which cannot persist any re-usable state in the context of the entire test session. In the case of Jest, its global variables are controlled by the test environment class.

Important caveats:

• runner realm can be also a top realm – for example, when Detox is started without the help of its own CLI and just by using test runner command.
• worker realm can be also a runner realm – for example, if the test runner does not support parallelism and runs everything in one thread.
• sandbox realm can be also a worker realm – in theory, a runner might not use any kind of sandboxing.

Hence, the entire distinction is conceptual above all. In practice, though, if we speak about the most common combination of detox-cli > jest-cli > ..worker.. > ..sandbox.., they indeed will be running as separate realms and processes in the operating system.

Resolving Detox config only once

This is very important because it eliminates the problem of multiple sources of truth, and reduces the time window "when Detox config is not resolved yet" – that reduces the number of assumptions in the code because the Detox state is almost never ambiguous.

[d4vidi]

Status: Requires discussion over what the best way to make progress here would be

@noomorph discuss over a meeting?

[d4vidi]

@noomorph following the last comment + today's planning: Are we cool regarding the next steps here?

[d4vidi]

Just went over this again. Excellent write up, @noomorph.