jest-mongodb icon indicating copy to clipboard operation
jest-mongodb copied to clipboard

Exception when using change streams

Open jmbldwn opened this issue 4 years ago • 3 comments

If I subscribe to a change stream, using mongoose's watch method, I encounter the following exception.

Is there a way to configure the mongo memory server to support this?

Exception has occurred: Error [ERR_UNHANDLED_ERROR]: Unhandled error. (MongoError: Storage engine does not support read concern: { readConcern: { level: "majority" } }
  at MessageStream.messageHandler (/Users/jim/development/.../node_modules/mongodb/lib/cmap/connection.js:263:20)
    at MessageStream.emit (events.js:315:20)
    at processIncomingData (/Users/jim/development/.../node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/Users/jim/development/.../node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at writeOrBuffer (internal/streams/writable.js:358:12)
    at MessageStream.Writable.write (internal/streams/writable.js:303:10)
    at Socket.ondata (internal/streams/readable.js:719:22)
    at Socket.emit (events.js:315:20)
    at addChunk (internal/streams/readable.js:309:12)
    at readableAddChunk (internal/streams/readable.js:284:9)
    at Socket.Readable.push (internal/streams/readable.js:223:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
    at TCP.callbackTrampoline (internal/async_hooks.js:131:14) {
  ok: 0,
  code: 148,
  codeName: 'ReadConcernMajorityNotEnabled'
})
    at ChangeStream.emit (events.js:304:17)
    at ChangeStream.<anonymous> (/Users/jim/development/.../node_modules/mongoose/lib/cursor/ChangeStream.js:41:51)
    at ChangeStream.emit (events.js:315:20)
    at processError (/Users/jim/development/.../node_modules/mongodb/lib/change_stream.js:572:38)
    at ChangeStreamCursor.<anonymous> (/Users/jim/development/.../node_modules/mongodb/lib/change_stream.js:436:5)
    at ChangeStreamCursor.emit (events.js:315:20)
    at /Users/jim/development/.../node_modules/mongodb/lib/core/cursor.js:343:16
    at /Users/jim/development/.../node_modules/mongodb/lib/core/cursor.js:736:9
    at /Users/jim/development/.../node_modules/mongodb/lib/change_stream.js:330:9
    at done (/Users/jim/development/.../node_modules/mongodb/lib/core/cursor.js:458:7)

jmbldwn avatar Apr 03 '21 14:04 jmbldwn

same issue

steven130169 avatar Apr 09 '21 09:04 steven130169

FYI, I was able to work around this problem, though I can't say for sure what exactly fixes this. Hopefully this will help others running into this.

  • I switched to using WiredTiger as the storage engine for mongodb-memory-server.
  • I modified my code that uses mongodb change streams to support shutting them down (for testing purposes)
  • I am using v7.0.0-beta38 of mongodb-memory-server

Here's the code I use to establish the replica set and subsequently shut it down:

    setupModelTest() {
        const { MongoMemoryReplSet } = require('mongodb-memory-server');             // use this if mongo not on host
        // const { MongoMemoryReplSet } = require('mongodb-memory-server-core');
        let replSet;
        let connection;

        beforeAll(async () => {
            replSet = await MongoMemoryReplSet.create({
                replSet: { storageEngine: 'wiredTiger' },
            });
            await replSet.waitUntilRunning();
            const uri = await replSet.getUri();
            connection = await mongoose.connect(uri, {
                useNewUrlParser: true,
                useCreateIndex: true,
                useFindAndModify: false,
                useUnifiedTopology: true,
                poolSize: 100
            });
        });

        afterAll(async () => {
            // shut down all change streams that may be open (.stopWatch() is implemented in every model)
            await require('.../models/xxx').stopWatch();
            await require('.../models/yyy').stopWatch();

            await connection.disconnect();
            await replSet.stop();
        });
    },

jmbldwn avatar Apr 09 '21 17:04 jmbldwn

FYI, I was able to work around this problem, though I can't say for sure what exactly fixes this. Hopefully this will help others running into this.

* I switched to using WiredTiger as the storage engine for mongodb-memory-server.

* I modified my code that uses mongodb change streams to support shutting them down (for testing purposes)

* I am using v7.0.0-beta38 of mongodb-memory-server

Here's the code I use to establish the replica set and subsequently shut it down:

    setupModelTest() {
        const { MongoMemoryReplSet } = require('mongodb-memory-server');             // use this if mongo not on host
        // const { MongoMemoryReplSet } = require('mongodb-memory-server-core');
        let replSet;
        let connection;

        beforeAll(async () => {
            replSet = await MongoMemoryReplSet.create({
                replSet: { storageEngine: 'wiredTiger' },
            });
            await replSet.waitUntilRunning();
            const uri = await replSet.getUri();
            connection = await mongoose.connect(uri, {
                useNewUrlParser: true,
                useCreateIndex: true,
                useFindAndModify: false,
                useUnifiedTopology: true,
                poolSize: 100
            });
        });

        afterAll(async () => {
            // shut down all change streams that may be open (.stopWatch() is implemented in every model)
            await require('.../models/xxx').stopWatch();
            await require('.../models/yyy').stopWatch();

            await connection.disconnect();
            await replSet.stop();
        });
    },

Thanks , I will try .

steven130169 avatar Apr 12 '21 14:04 steven130169