Getting data from system collections

[RoeiA]

Hello,

I've been trying out this plugin and it's great! I'd like to get data from a system.profile collection in order to create an operation analysis dashboard in Kibana. When I try to do that, logstash gives me this message: D, [2015-08-16T12:17:14.914000 #30642] DEBUG -- : MONGODB | COMMAND | namespace=collection1.$cmd selector={:listCollections=>1, :cursor=>{}, :filter=>{:name=>{"$not"=>/system.|$/}}} flags=[] limit=-1 skip=0 project=nil | runtime: 32.0001ms

So system collections are filtered out. The code responsible for that is in the ruby mongo driver: https://github.com/mongodb/mongo-ruby-driver/blob/15f5707794e852a8124d8d9d89019223e0183ebe/lib/mongo/operation/commands/list_collections.rb#L49

Is there any way to get around that in your plugin?

Thanks.

[phutchins]

Thanks for the input @RoelA

I see exactly what you mean. I'm contemplating a way to get around this now...

[phutchins]

@RoeiA I haven't had much of a chance to take a stab at this. Do you have any ideas on the best way we could accomplish this? I'd be glad to look at a merge request and help you work through it.

[oiahoon]

so? is there any work around for system.profile, or maybe we should build a exec

[sharmagaurav602]

Has anyone found a workaround for this ?

[matiasperrone]

Yes I used both mongosql and a JDBC driver from DbSchema (check: https://dbschema.com/jdbc-driver/MongoDb.html). Notice that mongosql generates "subtables" for the array fields.

I prefered the JDBC driver due to the fact that almost any shell query or function would work, like: db.MyCollection.find()

I hope it helps.

[sharmagaurav602]

Thanks @matiasperrone , will try that .