logstash-input-mongodb icon indicating copy to clipboard operation
logstash-input-mongodb copied to clipboard

Failing on document - Invalid ObjectID

Open jporter-dev opened this issue 9 years ago • 1 comments

I'm working on using this input gem to pull data from a MongoDB instance (that I don't own or manage) and sync it up with Elasticsearch. This input seems to do just that, but I'm having an issue actually fetching the data.

I get the following error on all objects:

MongoDB Input threw an exception, restarting {:exception=>#<BSON::ObjectId::Invalid: '[domain name or MD5]' is an invalid ObjectId.>, :level=>:warn, :file=>"logstash/inputs/mongodb.rb", :line=>"346", :method=>"run"}

I believe the error is because the IDs in this mongo collection are domains, and another collection uses md5 hashes as the ID. Is there any way to get around this or fix it?

I wanna say I think it's [https://github.com/phutchins/logstash-input-mongodb/blob/master/lib/logstash/inputs/mongodb.rb#L227](Line 227) that's erroring, but I could be wrong. I can post my logstash config if needed. Any help would be greatly appreciated!

jporter-dev avatar Feb 03 '16 20:02 jporter-dev

Also, looking into this a little more - correct me if I'm wrong - it doesn't appear that setting generateId to true does anything. Could enabling that potentially solve my problem?

jporter-dev avatar Feb 03 '16 21:02 jporter-dev