elasticsearch-river-mongodb
elasticsearch-river-mongodb copied to clipboard
richardwilly98
Loses sync regularly
I'm dropping / creating / refilling collections, and ElasticSearch constantly gets out of sync with the river.
Are there any recommendations or known forbidden actions?
Dropping doesnt work well for me. #464 To get around it, turn off indexOption: dropCollection and instead use remove() for each collection(this is followed through properly by ES in my experience). Aside from that, everything else seems to work well. If you are still out of sync, make sure your mongo topology is good(i.e, if you use shards, make sure you point elasticsearch to a mongo querier so it knows all shards. if you use replication, make sure all host names are set.). Also, make sure your ES topology is robust(avoids splitbrains, sufficient master nodes, good connectivity, good timeout values etc).
You mean I must not drop collection from mongo?
P.S. I don't have shards and my db is relatively small.
yes that is correct. Also dont use the dropCollection option when creating the river.
Ah I didn't know such river option even exists :)
It's possible, but rather cumbersome to avoid drop collection in my scenario. Why does it break the river? The oplog does not log it correctly?
No idea. No one responded to my ticket so I'm as confused as you are. If you are importing only one collection, it might work fine. But I recommend you test it for yourself and let me know if you find something else.
@iliakan if you are done, please close the issue. you can reopen when you have another concrete question.