idx icon indicating copy to clipboard operation
idx copied to clipboard

Composite sorted indexes?

Open julienfantin opened this issue 4 years ago • 2 comments

Let's say we added a :gender key to every user from the README:

[{:name "Alice", :age 42, :gender :f}
 {:name "Bob", :age 30, :gender :m}
 {:name "Barbara", :age 12, :gender :f}
 {:name "Jim", :age 83, :gender :m}]

A query such as males over 30, could be answered efficiently with this kind of structure:

;; NOTE users are sorted within their gender group

{:f [{:name "Barbara", :age 12, :gender :f}
     {:name "Alice", :age 42, :gender :f}]
 :m [{:name "Bob", :age 30, :gender :m}
     {:name "Jim", :age 83, :gender :m}]}

But I couldn't quite figure out if this was supported in the current api?

julienfantin avatar Oct 23 '20 21:10 julienfantin

You are right, this is not possible at the moment with the current api.

I would be interested in looking at use cases like this, but wonder whether its veering into index composition and query engine territory.

wotbrew avatar Oct 28 '20 09:10 wotbrew

You are right, this is not possible at the moment with the current api.

Thanks for clarifying.

I would be interested in looking at use cases like this, but wonder whether its veering into index composition and query engine territory.

Indeed, the issue's wording probably gave that away! Though these use-cases can go a really long way with just the ability to compose nesting and ordering, i.e. define different pivots for range scans.

I haven't dug into the internals so I'm not sure my mental model matches the implementation but it seemed like match already had the machinery required to control the index's nesting? If so it seems all that would be required here is a way to pass in some sort of comparator and keyfn to index which would be used with sorted-*-by when creating leaves in the index tree?

julienfantin avatar Oct 28 '20 17:10 julienfantin