MongoNotConnectedError on test using mongodb-memory-server and jest

[root-io]

trafficstars

Do you want to request a feature or report a bug? bug

What is the current behavior?

    MongoNotConnectedError: MongoClient must be connected to perform this operation

      at getTopology (node_modules/mongoose/node_modules/mongodb/src/utils.ts:451:9)
      at Collection.find (node_modules/mongoose/node_modules/mongodb/src/collection.ts:762:18)
      at Collection.findOne (node_modules/mongoose/node_modules/mongodb/src/collection.ts:740:17)
      at NativeCollection.<computed> [as findOne] (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:202:33)
      at NodeCollection.findOne (node_modules/mquery/lib/collection/node.js:34:21)
      at model.Query.Object.<anonymous>.Query.findOne (node_modules/mquery/lib/mquery.js:2021:20)
      at model.Query.<anonymous> (node_modules/mongoose/lib/query.js:2435:22)
      at model.Query._wrappedThunk [as _findOne] (node_modules/mongoose/lib/helpers/query/wrapThunk.js:27:8)
      at node_modules/kareem/index.js:372:33

If the current behavior is a bug, please provide the steps to reproduce.

The error happen on one test out of 7 and sometime it passes so I guess their is a race condition somewhere. Maybe related to #11445

What is the expected behavior? no error

What are the versions of Node.js, Mongoose and MongoDB you are using? Note that "latest" is not a version.

jest setupFilesAfterEnv (if you have suggestion to improve this part please tell me):

import mongoose from "mongoose";
import { MongoMemoryReplSet } from "mongodb-memory-server-core";

let replicaSet: MongoMemoryReplSet;

jest.setTimeout(30000);

beforeAll(async () => {
  replicaSet = await MongoMemoryReplSet.create({
    replSet: { storageEngine: "wiredTiger" },
  });
  const uri = replicaSet.getUri();
  await mongoose.connect(uri);
});

afterEach(async () => {
  const collections = await mongoose.connection.db.collections();
  for (const collection of collections) {
    await collection.deleteMany({});
  }
});

afterAll(async () => {
  await mongoose.connection.db.dropDatabase();
  await mongoose.disconnect();
  await replicaSet.stop();
});

Please note I use a change stream UserModel.watch().on("change" ...)

mongoose 6.2.10 mongodb 4.4.13 mongodb-memory-server-core 8.4.2 jest 27.5.1

[IslandRhythms]

https://mongoosejs.com/docs/jest.html

[root-io]

https://mongoosejs.com/docs/jest.html

I already followed this doc before posting my issue, I don't use timers in my code, is this an issue with the Jest config jest.setTimeout(30000);?

[IslandRhythms]

Re-reading your code, I think the problem is that you're trying to run a create operation while not connected to the database. Try connecting to the database first. If you think you are connecting to the database, add some connection event listeners with console.log()s to see when exactly it connects.

https://mongoosejs.com/docs/connections.html#connection-events

[root-io]

Running only the failed test with debug true and logged connected, disconnected and close events, everything looks ok but I still have the MongoNotConnectedError

See logs

  ● Console

    console.debug
      connected

      at NativeConnection.<anonymous> (src/jest.setup.ts:9:11)

    console.info
      Mongoose: users.findOne(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: users.insertOne(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: users.createIndex(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: tokens.createIndex(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: tokens.insertOne(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: users.findOne(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: tokens.findOne(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: users.findOne(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: tokens.findOne(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: users.updateOne(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: tokens.findOneAndDelete(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: tokens.findOne(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.info
      Mongoose: users.findOne(xxx)

      at NativeCollection.Object.<anonymous>.NativeCollection.$print (node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:263:11)

    console.debug
      disconnected

      at NativeConnection.<anonymous> (src/jest.setup.ts:13:11)

    console.debug
      close

      at NativeConnection.<anonymous> (src/jest.setup.ts:17:11)

Edit: I found a way to fix the issue, all my tests pass now:

afterEach(async () => {
-  const collections = await mongoose.connection.db.collections();
-  for (const collection of collections) {
-    await collection.deleteMany({});
-  }
+  await mongoose.connection.db.dropDatabase();
});

[sibelius]

where is the root cause of the bug?

is it mongodb core? is it on mongoose? is it on mongo memory server? is it on jest ?

[vkarpov15]

No idea. @sibelius are you running into a similar issue with setupFilesAfterEnv? Looks like Jest keeps on producing new ridiculous antipatterns for us to work around :disappointed:

[sibelius]

my pattern is similar to this one https://github.com/shelfio/jest-mongodb

I think this happen when we have some kind of memory leak in tests, and this happens a lot on jest

[root-io]

Fixed following: https://github.com/nodkz/mongodb-memory-server/issues/629#issuecomment-1092738978=

[vkarpov15]

Thanks @nodkz :+1:

@root-io I would be remiss if I didn't warn you about the dangers of testing with Jest. We highly recommend using any other test runner. Jest is not a test framework, it is an alternative JavaScript runtime. Testing an app with Jest but running it in Node is like testing the app in the browser but running it in Node - you'll get failures in test that don't happen in prod, and failures in prod that don't happen in test.

[sibelius]

I would avoid using replicaset on tests

this is what we run on afterAll

afterAll(disconnectMongoose);

export async function disconnectMongoose() {
  await mongoose.disconnect();
  // dumb mongoose
  mongoose.connections.forEach((connection) => {
    const modelNames = Object.keys(connection.models);

    modelNames.forEach((modelName) => {
      delete connection.models[modelName];
    });

    const collectionNames = Object.keys(connection.collections);
    collectionNames.forEach((collectionName) => {
      delete connection.collections[collectionName];
    });
  });

  // comment on v6
  if (mongoose.modelSchemas) {
    const modelSchemaNames = Object.keys(mongoose.modelSchemas);
    modelSchemaNames.forEach((modelSchemaName) => {
      delete mongoose.modelSchemas[modelSchemaName];
    });
  }

  mongoose.deleteModel(/.+/);
}

[vkarpov15]

@sibelius are you using Jest fake timers? That's a likely culprit, MongoDB Node driver's internal replica set monitoring logic makes heavy use of setTimeout() and setInterval().

[sibelius]

we are not using jest fake timers

[vkarpov15]

@sibelius I'm unable to repro, can you take a look at this GitHub repo and see if you can modify it to repro the issue you're seeing? https://github.com/vkarpov15/mongoose-11592

Here's the output I get:

$ npm test

> test
> jest ./test.js

 PASS  ./test.js
  mongoose
    ✓ create a doc (42 ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        0.419 s, estimated 1 s
Ran all test suites matching /.\/test.js/i.

[sibelius]

For me it is working well

[github-actions[bot]]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days

[github-actions[bot]]

This issue was closed because it has been inactive for 19 days and has been marked as stale.