roaring-rs icon indicating copy to clipboard operation
roaring-rs copied to clipboard

Add iter ops

Open saik0 opened this issue 3 years ago • 2 comments

Closes #58 Closes #109 Fixes #57

Adds operators to iterators of bitmaps

TODO

  • [ ] Treemap ops
  • [ ] Tests
  • [ ] Examples in docs

saik0 avatar Jan 12 '22 23:01 saik0

I see value in exposing an API for lazy ops regardless of whether or not we expose the iterator extensions that use them internally. To generalize to any kind of aggregation the user might need to do. The intuition is to have an API that allows users to do lazy operations with a newtype, that gets "repaired" (to use the same term as other lang impls) when it's unwrapped.

Thinking out loud in pseudo-rust about the shape of the API, not the names.

struct Lazy {
  bitmap: RoaringBitmap
}

impl BitOrAssign<&Lazy> for Lazy {
 // .. do a lazy or
}

impl BitXorAssign<&Lazy> for Lazy {
 // .. do a lazy xor
}

impl RoaringBitmap {
  fn into_lazy(self) -> Lazy
}

impl Lazy {
  /// ensure all the stores are the correct type before unwrapping (repairAfterLazy in other langs)
  fn into_bitmap(self) -> RoaringBitmap
}

fn main() -> {
  // Roughly equivalent to current IterExt (this does not use Cow)
  let bitmaps: Vec<RoaringBitmap> = todo!();
  bitmaps.into_iter().map(RoaringBitmap::into_lazy).reduce(|acc, other| { acc |= other; acc }).into_bitmap();

  // Now do it in parallel!
  use rayon::prelude::*;
  let more_bitmaps: Vec<RoaringBitmap> = todo!();
  bitmaps.into_par_iter().map(RoaringBitmap::into_lazy).reduce(|acc, other| { acc |= other; acc }).into_bitmap();
}

I'd need to think more about what it might look like for borrowed bitmaps. 🤔

saik0 avatar Jan 19 '22 21:01 saik0

To generalize to any kind of aggregation the user might need to do. The intuition is to have an API that allows users to do lazy operations with a newtype, that gets "repaired" (to use the same term as other lang impls) when it's unwrapped.

I like the idea, but we must absolutely make sure to document it enough. That's not clear what a Lazy type does. I like the idea of wrapping the bitmaps instead of methods to do lazy operations on them. I like the Lazy wrapping type as it look a lot like the Wrapping struct of the std.

Kerollmops avatar Jan 20 '22 09:01 Kerollmops