elasticsearch-river-mongodb icon indicating copy to clipboard operation
elasticsearch-river-mongodb copied to clipboard

does the river use a "notimeout" cursor?

Open derEremit opened this issue 10 years ago • 2 comments

I'm experimenting with switching to wiredtiger db engine an just read that with wiredtiger there are serious performance issues with using notimeout cursors.

without looking at the code it seems to me that this could be what the river could be using to "hook" to the oplog

derEremit avatar Jun 30 '15 20:06 derEremit

Yes, it does, but I'm not sure the performance issue with notimeout applies to the oplog, as opposed to other collections, perhaps because it's not the only flag set on the oplog cursor:

        int options = Bytes.QUERYOPTION_TAILABLE | Bytes.QUERYOPTION_AWAITDATA | Bytes.QUERYOPTION_NOTIMEOUT
        // Using OPLOGREPLAY to improve performance:
        // https://jira.mongodb.org/browse/JAVA-771
                | Bytes.QUERYOPTION_OPLOGREPLAY;

        DBCursor cursor = oplogCollection.find(indexFilter).setOptions(options);

We migrated to wiredtiger some weeks ago, and haven't seen any problems with the river being able to follow the oplog.

kdkeck avatar Jul 01 '15 20:07 kdkeck

I know this is an old thread, but I was wondering if you could tell me what versions of MongoDB, ES, and the river you were using that you were successfully able to use the river?

iprudhomme avatar Apr 08 '16 18:04 iprudhomme