thinky icon indicating copy to clipboard operation
thinky copied to clipboard

Smarter `filter` and `orderBy` index optimizations

Open marshall007 opened this issue 9 years ago • 4 comments

This PR exposes a new method Model::ensureCompoundIndex and in using it, allows thinky to optimize multi-field filter and orderBy queries using compound indexes.

let Person = thinky.createModel('person', {
  id: String,
  first_name: String,
  last_name: String,
  active: Boolean
})

Person.ensureCompoundIndex('full_name', [ 'first_name', 'last_name' ])

Person.filter({ first_name: 'Foo', last_name: 'Bar', active: true })
// -> .getAll([ 'Foo', 'Bar' ], { index: 'full_name' }).filter({ active: true })

Person.orderBy('first_name', 'last_name')
// -> .orderBy({ index: 'full_name' })

It might be nice if we detected implicit vars passed to the ensureIndex and automatically pulled the field name(s) from the query object. That way we wouldn't need a separate method and index optimization would automatically work for index functions like r.row('field') and [ r.row('field1'), r.row('field2') ].

marshall007 avatar May 12 '16 21:05 marshall007

The PR looks good to me. I'm a bit worried about the direction we are going though.

One of the nice thing about ReQL is that you know what RethinkDB is using. Indexes have to be explicitly used. I added this filter optimization at the beginning, but I'm not sure if someone is actually using it.

@marshall007 -- would you use filter instead of getAll knowing that filter wiill be transformed in a getAll?

neumino avatar May 14 '16 18:05 neumino

I think this PR is important, if thinky can help optimize queries then the developer has less to think about at least for filters and order by. Of course this should be explicitly requested, toggling a parameter / or calling a specific function like ensureCompoundIndex. I would give it a go! :)

fenos avatar Jul 06 '16 23:07 fenos

would you use filter instead of getAll knowing that filter will be transformed in a getAll?

@neumino sorry for the delay in following up with this, but yes absolutely! I have several apps where I end up duplicating similar logic for building an efficient query based on the specified query string parameters.

marshall007 avatar Jul 07 '16 19:07 marshall007

Also, in terms of the overall direction here I think optimizations like these are a completely reasonable and often expected feature of ORMs. I agree with @fenos that by explicitly defining by indices in the ORM I should reasonably expect it to optimize my queries automatically whenever possible.

marshall007 avatar Jul 07 '16 19:07 marshall007