elasticsearch-river-mongodb icon indicating copy to clipboard operation
elasticsearch-river-mongodb copied to clipboard

River does not work with external mongodb replica set

Open phongmh307 opened this issue 9 years ago • 2 comments

River works perfectly on my local environment, but when i deployed it to my production environment which using an external mongodb replica set, it seems to unable to connect.

All port are open. I can connect to ES from my mongo server and I also can connect to mongodb from ES server via mongo

This is the TRACE:

[2015-04-15 03:24:53,173][INFO ][river.mongodb            ] [Ocean] Creating MongoClient for [[130.x.x.x:27017, 104.x.x.y:27018, 104.x.x.x:27000]]
[2015-04-15 03:24:53,324][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:24:53,325][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:25:33,327][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:25:33,328][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:26:13,329][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:26:13,330][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:26:53,331][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:26:53,332][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:27:33,332][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:27:33,333][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:28:13,334][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:28:13,334][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:28:53,335][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:28:53,336][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:29:33,337][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:29:33,337][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:30:13,338][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:30:13,339][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:30:53,340][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:30:53,341][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}
[2015-04-15 03:31:33,342][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] Found admin database
[2015-04-15 03:31:33,342][TRACE][org.elasticsearch.river.mongodb.MongoConfigProvider] About to execute: { "serverStatus" : 1 , "asserts" : 0 , "backgroundFlushing" : 0 , "connections" : 0 , "cursors" : 0 , "dur" : 0 , "extra_info" : 0 , "globalLock" : 0 , "indexCounters" : 0 , "locks" : 0 , "metrics" : 0 , "network" : 0 , "opcounters" : 0 , "opcountersRepl" : 0 , "recordStats" : 0 , "repl" : 0}

phongmh307 avatar Apr 15 '15 03:04 phongmh307

Had exactly the same problem. After couple days of investigation, I could figure it out (Ubuntu 14.04.1 LTS, MongoDb 2.6.5, river 2.0.2, ES 1.3.5). Maybe, it'll work for you (or save someone's time) too.

  1. connect to your mongo shell
  2. run rs.conf()
  3. Check whether host in rs.conf() output matches one, you use in river connection settings

I just created mongo replica set via rs.initiate(), and it seems by default it uses hostname fqdn. After I updated my replica set hostname and recreated river, I got it working. So, you need host in your river configuration match with host from rs.conf() of mongodb

Hope this help

m-teslya avatar May 10 '15 17:05 m-teslya

It doesn't work for me.

akluffy avatar Jul 12 '15 05:07 akluffy