storage: generic nullable block encoding

[skyzh]

Currently, the nullable encoding is done by using a bitmap, and is only implemented for primitive types.

https://github.com/risinglightdb/risinglight/blob/7b5bc7a1a258afd64a483990ca95f4c799482791/src/storage/secondary/block/primitive_nullable_block_builder.rs#L18

Indeed, such logic can be done in a more generic way. We can wrap any BlockIterator to become a nullable encoding.

pub struct NullableBlockBuilder<B: BlockBuilder> {
  inner: B,
  bitmap: BitVec
}

To achieve this, we can:

• [ ] separate nullable encoding from primitive nullable encoding.
• [ ] use the generic nullable encoder for char and blob.
• [ ] to avoid branch prediction for Option::unwrap, we can make a new pair of traits: NonNullableBlockBuilder and Iterator, which accepts &A::Item instead of Option<&A::Item>.

[wxmbugu]

I'm a beginner and would like to give this a try.

[skyzh]

Welcome to have a try!

[eliasyaoyc]

it seems like?

pub struct NullableBlockBuilder<T: PrimitiveFixedWidthEncode, B: BlockBuilder<T::ArrayType>> {
	inner: B,
	bitmap: BitVec,
	phantom: PhantomData<T>,
}

impl<T: PrimitiveFixedWidthEncode, B: BlockBuilder<T::ArrayType>> BlockBuilder<T::ArrayType>
    for NullableBlockBuilder<T, B>
{
    fn append(&mut self, item: Option<&T>) {
        if item.is_none() {
            self.inner.append(T::DEFAULT_VALUE);
            self.bitmap.push(false);
        } else {
            self.inner.append(item);
            self.bitmap.push(true);
        }
    }

    fn get_target_size(&self) -> usize {
        self.inner.get_target_size()
    }

    fn finish(self) -> Vec<u8> {
        self.inner.finish()
    }
}

impl<T: PrimitiveFixedWidthEncode, B: BlockBuilder<T::ArrayType>> NullableBlockBuilder<T, B> {
    pub fn new(builder: B) -> Self {
        Self {
            bitmap: BitVec::with_capacity(builder.get_target_size()),
            inner: builder,
            phantom: PhantomData,
        }
    }
}

[skyzh]

This design looks good!

[unconsolable]

@eliasyaoyc Hi, are you working on this issue? I'm trying to work on it a few days ago. So would you like to wait for a while?