logstash-input-mongodb icon indicating copy to clipboard operation
logstash-input-mongodb copied to clipboard

Missing data problem and data update question

Open TaurusD opened this issue 7 years ago • 3 comments

Hi, I’m using logstash-input-mongodb to import my data from mongodb to ELK. But when it was done, I found that there is one document missing (I have 17781 docs in mongodb but only got 17780 docs in ELK).

I used the “normal” way to install the plugin through command bin\logstash-plugin install logstash-input-mongo. So can anyone help me on this problem?

Also, I found that when there is a change on the source db in mongo, this plugin can not detect it and import new data. And I didn’t found any schedule settings (like jdbc input plugin) which allow me to call my data regularly. So can anyone tell me how to keep data updated with mongodb using this plugin?

Thank you!

TaurusD avatar Aug 01 '17 03:08 TaurusD

+1 but I have 399712 docs in mongodb but only got 399700 docs in ELK

alejandrobarrios07 avatar Aug 01 '17 17:08 alejandrobarrios07

now I had another try, which only 24 docs in mongo and 23 docs in ELK. can anyone help us? @phutchins

I sorted by _id and found it was the first doc (smallest _id) didn't import into ELK

TaurusD avatar Aug 03 '17 07:08 TaurusD

What's been done in this issue's regard? Its been a long time since this issue has been reported and still I am also getting the same problem i.e First record goes missing in ELK.

topgun743 avatar Jan 09 '19 11:01 topgun743