River not synchronizing.

[fuqqer]

Trying to create a river and no documents are getting synced. From my elasticsearch host I type: curl -XPUT localhost:9200/_river/toku/_meta -d '{ "type": "mongodb", "mongodb": { "servers": [ { "host": "server1.mydomain.com", "port": 27017 }, { "host": "server2.mydomain.com", "port": 27017 }, { "host": "server3.mydomain.com", "port": 27017 } ], "db": "inbound_queue", "collection": "metadata", "options": { "secondary_read_preference": true }, "gridfs": false, "credentials": [ { "db": "local", "user": "myuser", "password": "mypassword" }, { "db": "admin", "user": "myuser", "password": "mypassword" } ], "index": { "name": "metadata", "type": "toku" } } }'

I visit myelasticsearchhost:9200/_plugin/river-mongodb and see the river as: Settings:

{ "type": "mongodb", "mongodb": { "servers": [ { "port": 27017, "host": "server1.mydomain.com" }, { "port": 27017, "host": "server2.mydomain.com" }, { "port": 27017, "host": "server3.mydomain.com" } ], "credentials": [ { "password": "mypassword", "user": "myuser", "db": "local" }, { "password": "mypassword", "user": "myuser", "db": "admin" } ], "options": { "secondary_read_preference": true }, "gridfs": false, "index": { "name": "metadata", "type": "toku" }, "collection": "metadata", "db": "inbound_queue" } }

I tried tunning on debug info in elasticsearch and in the log when I click start, all I see are the settings listed in the river page. I see no errors about connecting to the mongo servers and I see nothing in the mongo logs either.

[ewgRa]

Can be related to #470 and big chance that it is connection problem, if there is no any records in logs.

Check that with credentials that you have you can connect to mongodb from server, where is ES server running.

I mention that you have database inbound_queue, but credentials pointed to local and admin database only. Maybe this is a reason. Also try connect to only one mongodb server, also as check that you have proper versions of MongoDB and ES river.

[fuqqer]

I am able to connect to my replica set using mongo and the credentials supplied to the river from my ES server. I also tried to create the river with only one mongo server to connect to and I still see no activity. I did a tcpdump while connecting with the mongo client and see the conversation happen between my ES server and the mongo server. However, when I create the river and start it, I see NO traffic on port 27017...any other suggestions on how to get the elasticsearch river to connect to a replica set?

I see nothing in the ES logs about the connection either.

[ewgRa]

I see nothing in the ES logs about the connection either.

if I'm right in #470, logs will be empty. Maybe if you stop river - there is will be something in logs. Try make your river config as light as possible, install mongodb without credentials and so on. If river will start working - you miss something in checks of connection and so on. If this is not help - try to debug river code sources or wait author of code, maybe he can help you somehow.

[Perchework]

I had the same probleme, and to resolve it, I change an entry in the /etc/hosts of the mongodb server :

not working /etc/hosts: 127.0.0.1 server1.mydomain.com

working /etc/hosts 192.168.0.5 server1.mydomain.com (change to the ip of the interface where elasticsearch call to mongodb)

[fuqqer]

Adding a host entry didn't help. :(

creating/starting/stopping the river while running tcpdump -vv|grep 27017 shows no traffic on port 27017. I'll try a local install in the meantime to test. Does es need to be running or the river created as a particular user?

[ewgRa]

You have an error in configuration.

"index": { "name": "metadata", "type": "toku" },

must be outside of "mongodb", at one level with "type": "mongodb", check examples https://github.com/richardwilly98/elasticsearch-river-mongodb/wiki

[marlonoso]

I got the exact same issue, and my configuration is OK. I've been struggling for days with this I even tryed different servers and elastic hosts It works only when elastic and mongodb are in the same server (host: localhost)

curl -XPUT 'http://localhost:9200/_river/mongodb/_meta' -d '{ "type": "mongodb", "mongodb": { "servers": [ { "host": "a.some.public.ip", "port": 27017 } ], "db": "social_client", "collection": "social_mention_event" },

"index": {
  "name": "client",
  "type": "social"
}

}'

[ewgRa]

Check all messages here and try it. It hard to say where is a problem exactly. If it is work on localhost, but don't want work on another hosts - that there is connection problems. Try connect directly as described. Reason can be firewalls, wrong credentials, no attention to details and so on.

[marlonoso]

Before switching to a replication-from-remote-server-to-my-elastic-host-approach, I will log here my issue:

• Remote Mongo server does not use authentication
• I can telnet a.some.public.ip 27017
• I'm doing a tcpdump port 27017 on a.some.public.ip
• It shows a little traffic when restarting ES after setting the afore mentioned config. Then traffic stops
• Meanwhile, ES logs:

[2015-03-19 14:42:30,403][INFO ][http ] [Vampire by Night] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.247.68:9200]} [2015-03-19 14:42:30,403][INFO ][node ] [Vampire by Night] started [2015-03-19 14:42:30,922][INFO ][gateway ] [Vampire by Night] recovered [2] indices into cluster_state [2015-03-19 14:42:31,439][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] MongoDB River Plugin - version[2.0.6] - hash[3c38a8a] - time[2015-03-10T03:06:51Z] [2015-03-19 14:42:31,441][INFO ][river.mongodb.util ] setRiverStatus called with mongodb - RUNNING [2015-03-19 14:42:31,450][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] River mongodb startup pending [2015-03-19 14:42:31,455][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] Starting river mongodb [2015-03-19 14:42:31,456][INFO ][org.elasticsearch.river.mongodb.MongoDBRiver] MongoDB options: secondaryreadpreference [false], drop_collection [false], include_collection [], throttlesize [5000], gridfs [false], filter [null], db [social_client], collection [social_mention_event], script [null], indexing to [client]/[social] [2015-03-19 14:42:31,458][INFO ][river.mongodb ] [Vampire by Night] Creating MongoClient for [[66.45.229.186:27017]]

Then _curl -XGET 'http://localhost:9200/client/mapping' shows {"client":{"mappings":{}}}

No hope

[ewgRa]

Please, create new issue.

[ewgRa]

@fuqqer please, close this issue if river works

[fuqqer]

The river still does not work. :( There are NO hostname resolution issues, and no trouble connecting between the hosts, they are on the same physical switch on the same LAN and there are no firewalls between them. I see no traffic even trying to connect on port 27017 when starting the river.

To simplify, I'm connecting to a single replica set with three members, not a sharded cluster.

I should also mention I'm using ES 1.5, I have both tokumx 2.0, and mongo 2.6.8 clusters I've tried to connect to with the same symptom.

I have tested connectivity by installing mongo-clients on an ES host and running: mongo mongo1.mydomain.us/admin -u adminuser -p adminpassword mongo mongo1.mydomain.us/my_database -p dbpassword

The adminuser has a clusterAdmin role. The my_database user has a dbwrite role.

I feel like it's a syntax error in creating the river. Here is what I'm using to create the river:

curl -XPUT localhost:9200/_river/protect_results/_meta -d '{ "type": "mongodb", "mongodb": { "servers": [ { "host": "mongo1.mydomain.us", "port": 27017 }, { "host": "mongo2.mydomain.us", "port": 27017 }, { "host": "mongo3.mydomain.us", "port": 27017 } ], "db": "my_database", "collection": "my_collection", "options": { "secondary_read_preference": true }, "gridfs": false, "credentials": [ { "db": "local", "user": "adminuser", "password": "adminpassword" }, { "db": "my_database", "user": "dbuser", "password": "dbpassword" }, { "db": "admin", "user": "adminuser", "password": "adminpassword" } ] }, "index": { "name": "my_collection", "type": "my_database" } } '

Any ideas?

[fuqqer]

*bump