fdb-document-layer icon indicating copy to clipboard operation
fdb-document-layer copied to clipboard

Use index for $in queries

Open zorkian opened this issue 5 years ago • 2 comments

It's possible I'm doing something wrong, but it appears that indexes are not used when performing $in queries. Here's an example Mongo query:

>>> import pymongo
>>> db = pymongo.MongoClient('mongodb://10.101.0.44:27016/')['discord']
>>> db.users.find({ '_id': { '$in': [ 3187217, 4248493 ] } })

On the backend trace, this is emitted:

<Event Severity="10" Time="1562606405.236166" Type="SlowQuery" ID="0000000000000000"
  Database="discord" Collection="users" Query="{ _id: { $in: [ 3187217, 4248493 ] } }"
  Plan="{ type: &quot;filter&quot;,
          source_plan: { type: &quot;table scan&quot; },
          filter: &quot;ANY(ExtPath(_id) matching OR(EQUALS('3187217'), EQUALS('4248493')))&quot; }"
  Machine="0.0.0.0:27016" LogGroup="default" />

I verified, just to be sure, that the index exists:

>>> import pprint
>>> pprint.pprint(db.users.index_information())
{u'_id_': {u'key': [(u'_id', 1)],
           u'metadata version': 1,
           u'ns': u'discord.users',
           u'status': u'ready',
           u'unique': True}}

My expectation in this case would be that the document layer would do two indexed lookups and return the results, and it shouldn't need to do a table scan to accomplish this? I read through the differences and caveats but don't see this documented anywhere.

zorkian avatar Jul 08 '19 18:07 zorkian

I read through the code for the query planner, this is definitely just unsupported at present. Any plans to implement this? It seems like it shouldn't be that difficult, but I suspect there are things I don't understand about the underlying system.

zorkian avatar Jul 09 '19 21:07 zorkian

I read through the code for the query planner, this is definitely just unsupported at present. Any plans to implement this? It seems like it shouldn't be that difficult, but I suspect there are things I don't understand about the underlying system.

It's only for "_id" field. I've fixed this issue in my fork. But I've not time for separate fix.

actopolus avatar Nov 03 '19 19:11 actopolus