jest
jest copied to clipboard
jestjs
Allow to share global state between tests from globalSetup
🚀 Feature Proposal
Add a global property to the this of globalSetup and globalTeardown async functions that can be used to set global variables that can be accessed in all tests via global. The same global is shared across all tests.
Motivation
While jest was in the beginning used only for frontend testing, it has moved in the direction of becoming a general test framework. Especially for backend integration tests there is a tradeoff between test speed and departmentalization: Starting up a new backend instance for each individual test usually isn't feasible. Therefore most other test frameworks like mocha or jasmine provide possibilities to share state between tests, e. g. the backend instance. Usage examples include mocking http requests via nock.
Example
Let's assume an integration test that tests backend and database integration. The setup could look like this:
const backend = require('backend')
async function setupApp () {
await new Promise((resolve, reject) => {
backend.start().then((instance) => {
this.global.backend = instance
})
})
}
module.exports = setupApp
And using the global could be done like this:
const request = require('supertest')
test('should call out to that fancy other api', () => {
request(jest.globals.backend.url)
.post('/some-endpoint')
expect(200)
})
Pitch
As far as I know this change currently cannot be implemented outside of the main jest framework. Closest is an environment, but environments are sandboxed and do not share global state.
Open questions
How to best implement it?
I don't know the jest code well enough to have an idea how to best implement this. It might e. g. be easier to make the global available via global, or even jest.getGlobals().
Can we prevent misuse?
Sharing state between tests can lead to sideffects and random test breakage. One possible solution would be to make the jest.globals read-only, but I am not sure whether this is feasible without massively reducing which kind of objects can be stored.
First comments is that the setupfiles shouldn't assign to this. If we do this, I think the setup should return something, and we can assign that inside of jest itself (https://github.com/facebook/jest/issues/5731#issuecomment-385070715).
Also, I think anything assigned will need to be serializable, I don't think it's technically possible for it to be e.g. an instance of something (we need to send it to workers which is a separate node process)
I took the this pattern from environments. It seemed a bit odd to me, too, but would be consistent.
If the object needs to be serializable, then unfortunately a lot of merit of this feature would be lost. My main usecase would indeed be using nock which attaches itself to its processes http module and therefore needs to be called in the same process as the one where the backend is running. It would be possible to set up some helper though I guess that communicates via serializable data. In that case we are talking more about inter-worker-communication then mere globals.
Yeah, the envs are the ones that construct the global used in tests, so it makes sense that they have it. Doesn't mean it's a nice pattern, though 😀
Due to a hole in the sandbox (we give you the real core and native modules) nock should work. Note that it might break at any time, as that's a bug.
It should? Interesting, I'll give it another try. Last time I didn't get it to work. Basically what this feature proposal is about is providing a sanctioned way to do this.
I agree with @dbartholomae on this issue, I find it hard to recommend jest for all types of testing without the ability to share state between tests. I have a real usecase currently where the company I work for wanted to standardize our testing frameworks so I do to start using Jest over Mocha for my functional API testing for our react app. that was a mistake given that I have to fetch a new bearer token for every test file with no way of retaining that token to a variable "globally".
I also agree with this issue - my team is using Jest/Supertest to test APIs for a microservice, and external service dependencies are faked using node/http. The setup is fantastic other than we have to use --runInBand always because we can't simply reuse external fake processes across tests, and it's not practical to test a single A microservice instance with various B and C dependency service instances running on random ports because each test is run in a separate process and can't access global node/http fakes of B and C. I hope this helps illustrate the issue at a high level better.
My use case involves testing mobile devices with Appium. Without a global handle to the chromium webdriver (which connects to the device through the appium server and installs the app), each testfile must repeat this process of setup and teardown of the app. It adds up to 40 seconds for each testfile to go through this process. As it stands right now, I also have to --runInBand of course since otherwise the tests will all try to instantiate their own chromedriver connection at the same time.
I have seen some really gross workarounds to this problem that abstract the various tests in each testfile into regular js functions, and then make you call all the functions inside a single shell test.js file that contains the describe/it structure. I would really prefer not to do this since it breaks the ability to run a specific testfile on demand by passing the test as a CLI argument. :-(
Fans of this may like the newly opened Feature Request as seen right above :)
Allow module sandbox to be disabled via configuration https://github.com/facebook/jest/issues/8010
Just adding our use case:
We have an asynchronous initialization step that we need to do as a one time setup (and then expose the result to individual tests). The initialization is expensive and really should only happen once for the duration of the whole test run, but as it is, without runInBand, it's not possible.
+1 here, there's some backend setup we'd like to share across all suites
Note that you'll never be able to share things that are not json-serializable as we have no way of passing that to workers. But sharing e.g. URLs to services will work whenever we get around to this.
So things like chromedriver connections talked about above cannot be supported. Puppeteer deals with this through exposing a websocket: https://github.com/smooth-code/jest-puppeteer/blob/master/packages/jest-environment-puppeteer/src/global.js
This is a deal breaker, something has to be done for sure. Most of these apps take 20secs or more to bootstrap and having that happen for 30 or 50 times (once for each test file) is a big no no. It should only happen once as stated above. Can't Jest pass state to it's child processes or something along those lines. It'd be ok if all test files could just access even the master worker's global state.
No, that's not how communication between processes work: https://nodejs.org/api/child_process.html#child_process_subprocess_send_message_sendhandle_options_callback
It's a bit better with worker_threads for builtin primitives, but not much: https://nodejs.org/api/worker_threads.html#worker_threads_port_postmessage_value_transferlist
This isn't an API choice Jest has made, it's a fundamental technical limitation. Puppeteer allows connecting to a running instance through a websocket, you need to do something similar for whatever thing you're instantiating in a globalSetup.
can you give a working example please? Whatever workaround you have to pass around instances, can't it be integrated into Jest or at least documented in Jest docs.
I don't know in how many ways I can say this, but I'll try one last time: you cannot pass around instances, it's not possible.
And seeing as this issue is still open, we have no solution for passing anything else either. jest-puppeteer adds the WS url to process.env, which works. When we fix this issue, they'll have an API they can use, which will be documented. We'll not be documenting hacks. The API will allow you to pass strings, numbers, objects and other primitives (not sure about regex, we'll see), but not instances
Please don't keep asking about things I've stated multiple times are not possible, or I'll have to lock this issue
can you give a working example please?
I recommend asking on StackOverflow or our discord channel for help.
take puppeteer for example: https://jestjs.io/docs/en/puppeteer.html
module.exports = {
globalSetup: './setup.js',
globalTeardown: './teardown.js',
testEnvironment: './puppeteer_environment.js',
};
@SimenB would it be feasible to not use worker processes, given some flag?
Some problem,I need async get token before all tests running,but official docs say that
Note: Any global variables that are defined through globalSetup can only be read in globalTeardown. You cannot retrieve globals defined here in your test suites.
So I try use another way,custom testEnvironment,and It works.
@shengbeiniao that's for JSON-serializable stuff. That's not what we're trying to achieve here.
Guys, I saw this and hope it can achieve what we are trying to do here. It's being used in VueJS as well for handling SSR stuff whilst passing almost anything around.
https://www.npmjs.com/package/serialize-javascript
Couldn't we use use this to serialize things and then unserialize for use later in scripts. Therefore sharing state across processes using globalstate that has that limitation.
Is NYC's __coverage__ variable getting a free pass? I am able to get/set to it inside tests.
Seeing that this issue is open might make a visitor think this is not possible. As mentioned by @SimenB it is. For many use cases, including mine, passing a string from globalSetup to the test suites was all that I needed.
Here's a link to my project (see rules-test/tools/guarded-session.js especially):
https://github.com/akauppi/GroundLevel-es6-firebase-web/tree/master/rules-test
The API will allow you to pass strings, numbers, objects and other primitives (not sure about regex, we'll see), but not instances
That's good clarity. So I take there is no guarantee that the process.env will continue working, but there will be an API that does the same.
I think my desired use-case is probably not possible, but I figured I'd chime in here as well.
I need to run integration tests with a server that is instantiated in-memory. That means I'm doing something like this:
const server = await startServer()
And then I need to use that server instance over a bunch of my tests files. To make matters worse, that particular startServer function takes about 5 to 10 seconds to startup (this is non-negotiable).
If instances cannot be passed around, then it sounds like I'm out of luck. But I would really love it if someone could tell me that I am wrong and that a solution is just around the corner.
My use case is ts-jest wants to pass an Object of file content cache to global to allow each worker to access and use it
At the moment each worker performs readFileSync to access this file content cache from disk.
So I'm trying to understand - most of this discussion is how to handle non-serializable items, but where did we leave off on allowing serializable items from globalSetup? I have a similar case with above where I am unable to set machine ENV variables on the fly through a script or other (whether during globalSetup, or outside it before running Jest). I would be attempting to store a serializable string as a global variable for my tests to achieve:
- It is only initialized/stored once
- It is available to all tests
There's not a good solution currently for Jest as the testEnvironment config file runs multiple times (for every test file), but is able modify variables to be made available to all tests; globalSetup config file runs once as desired, but is unable to modify variables to be made available to all tests.
@adrianmcli @brianschardt
It looks like currently what we can do is something like the following if we want to avoid slow startServer() execution per test file
// entrypoint.test.js
beforeAll(async () => {
global.server = await startServer();
});
import './tests/my-first-test';
import './tests/my-second-test';
import './tests/other-test';
// ./tests/my-first-test.js
test('my test', () => {
expect(global.server.doSomeIntereaction()).toBe(...)
})
Of course, this does not come with parallelization.
Probably ideal solution would be launching startServer() as a separate process like jest-puppeteer does, but in that case I am not sure how we can easily swap mocked functions behind startServer() for each test case....
I think that #8708 would solve a bunch of problems people have that motivate this kind of state sharing. Instead of having to share something from the global setup down to the child processes, each of those child processes should (probably?) own one and only one of the resources. A puppeteer, a database connection, a booted server, etc, should be one per worker, not one global one per test run, and also not one per test file. That plays nicest with the automatic parallelization, keeps things fast, and I think is semantically sound with what Jest does already. Would that work for you folks and if so please thumbs up #8708!
Other resources like Kafka and ElasticSearch we use it in a multi-tenant fashion. But to do so we must pass some information like connection string, admin credentials, unique run id for build parallelization on the same host etc.
Right, that's already possible with globalSetup via the process.env hack described above. You can boot the global resource once in globalSetup, and then pass whatever string-serialized state is necessary down to the workers via process.env. That's not training wheels, that's a fundamental limitation of the parallelization Jest gives you, so I'm not sure what you mean by that. If the workers need some piece of state that is the same for each worker but different between them, like say a Kafka connection or an ElasticSearch connection locked to a namespace for that worker, you still need to boot that up per-worker. If you want to have workers be different tenants then they need to act differently, no?
I think we can say there are many levels and contexts of setup:
- per-invocation config (
globalSetup,globalTeardown) - per-worker (not existent, see #8708)
- per-suite (
setupFiles,setupFilesAfterEnv,beforeAll,afterAll) - per-test (
beforeEach,afterEach).
There is a process boundary between the per-invocation and per-worker layers that allows for parallelization and the Jest authors have said won't go away. That's the right move IMO. That means you can't ever pass real objects and especially not real open sockets down from the one top places to N inner places.
What I am talking about is giving Jest users the ability to run setup once for the worker instead of once per suite. Connecting to ES or Redis or what have you once per suite isn't the end of the world, but for services that don't have multi-tenancy built in like puppeteer, or for app-land code that is expensive to boot, it'd be nice to do it once per worker instead of once per suite. Say creating kafka topics for the worker to use, or creating a Postgres database like mycoolapp_test__${process.env.JEST_WORKER_ID}. Right now that's not really possible, and I think it'd make managing these backend services a lot easier.
I also think tearing down Kafka / ES from jest isn't the best idea -- teardowns are more best effort than gauranteed IMO. You can't really guarantee that the teardown runs because a dev could SIGTERM the process, or it could OOM, or whatever really. The best use of those backend services would be re-entrant and automatically massage whatever state is in them into the clean state necessary for the test.