CRoaring icon indicating copy to clipboard operation
CRoaring copied to clipboard

Implement Bit-sliced index for CRoaring

Open Smallhi opened this issue 2 years ago • 2 comments

There are implements for bit slice index in java and go. Do we need in C/C++?

I think implementing Bit-sliced index for CRoaring include these tasks:

  • support mutable Bit-sliced index in C
  • support RangeBitmap in C
  • A range Index demo for postgres using RangeBitmap in C
  • A mutable bit-sliced index demo for postgres using bit-sliced index in C
  • A range index demo for Apache Parequet file-format in C/C++
  • A mutable bit-sliced index demo for Apache Parequet file-format using Bit-sliced index in C

@lemire we've implemented mutable Bit-sliced index in C and using it for postgres. Due to there are two implements for bit-sliced index, (BSI and RangeBitmap), one is mutable but the performance is poor, the other is immutable ,but the performance is great. Could we make a decision or re-design it ?

cc @ richardstartin

refer to :

  1. https://richardstartin.github.io/posts/range-bitmap-index
  2. https://richardstartin.github.io/posts/range-predicates
  3. https://github.com/RoaringBitmap/RoaringBitmap/tree/master/bsi
  4. https://github.com/lemire/BitSliceIndex/issues/1

Smallhi avatar Feb 11 '23 03:02 Smallhi

Yes, a pull request to provide this functionality in C is invited.

lemire avatar Feb 11 '23 23:02 lemire

@Smallhi : I alos have this use case , any idea when would we be able to merge this implementation. Also i have a noob question : Why is bit sliced index dependent on Rangebitmap implementation ? What exactly is the difference between range index and bit slice index ?

patelprateek avatar Nov 06 '23 05:11 patelprateek