logstash-input-mongodb icon indicating copy to clipboard operation
logstash-input-mongodb copied to clipboard

how to move the last modified tuple/rows from mongodb to elasticsearch

Open monojitbarua opened this issue 6 years ago • 2 comments

new inserted rows are coming as mentioned in your rb file( :_id => {:$gt => last_id_object}). But is there any way to get the rows/tuples from the mongodb collection which are updated after insertion.

monojitbarua avatar Jan 08 '19 15:01 monojitbarua

From the README:

This was designed for parsing logs that were written into mongodb. This means that it may not re-parse db entries that were changed and already parsed.

...which, for me, makes it useless.

rsimonton avatar Jan 28 '20 18:01 rsimonton

@monojitbarua I was stuck in the similar problem as none of the available plugins/tools fully support elasticsearch >7.0 and lacks basic features. I end up writing simple python script to ingest data from mongo to elasticsearch. Feel free to use it https://github.com/hardiksondagar/mongodb-to-elasticsearch

hardiksondagar avatar Sep 21 '20 12:09 hardiksondagar