replicache icon indicating copy to clipboard operation
replicache copied to clipboard

Add support for other JavaScript types to indexes

Open aboodman opened this issue 2 years ago • 3 comments

This was just something we overlooked when first implementing. We would have to add some bits to the index value to distinguish types and decide on an ordering.

aboodman avatar Dec 13 '22 06:12 aboodman

The obvious type to add support for is number. However, the ordering we use is UTF8 strings. How do we encode an arbitrary number in a way that we preserve the ordering? This paper looks promising: https://www.zanopha.com/docs/elen.pdf

There is also the issue of how to deal with the secondary key in scan

arv avatar Jan 02 '23 10:01 arv

I'm not following. At the basic level we can clearly encode the number however we want and decode it to compare. Is the problem that you're trying to find a way that we can avoid that and compare as a string? That's an interesting problem. It seems similar to the fracdex problem?

aboodman avatar Jan 03 '23 00:01 aboodman

  1. The "value" is part of the key in the underlying B+Tree we use for the index.
  2. To get scan to iterate over the keys in the correct order we need to have a mapping from number to string that preserves the numeric order. If it was only ints we could zero pad the number for example.
  3. For scan we can pass the start key which is a tuple for index scans consisting of the [secondary: string, primary: string]. If the index is over numbers we would want to pass a number into secondary

arv avatar Jan 03 '23 09:01 arv