Daniel Lemire
Daniel Lemire
> However, there is a little problem, a memory alignment problem. In Rust (and other languages), it is undefined behavior to read a type that isn't well aligned in memory...
@Kerollmops Roaring bitmaps have a simple format that is easy to deserialize quickly, but if you are constantly grabbing bytes, and copying them into Roaring bitmaps data structures, it is...
Same functionality in C: https://github.com/RoaringBitmap/CRoaring/blob/140e6a8bd92395d5392b02266b533e178a3e00d2/include/roaring/roaring.h#L521
In C it is called a 'frozen view'. The C function can be called from Go (we have a package that does so).
> I looked at the function, and indeed there is a lot of potential unaligned memory reads in it. How confident are you about this statement? The frozen serialization and...
> Ok, so what I understand here is that any C library that needs to use this bitmap view system must also be compiled with the memory alignment sanitizer and...
> Yes, that would definitely be our use case (currently we are doing the mentioned check before execution the contains), so we thought to integrate it within the bitmap. Well,...
I would agree that such a gain makes a few extra lines of code worth it.
> we are getting slightly worse performance when including the check you proposed Introducing more branch paths can make the results worse. But you should be concerned with the quality...
@LucaCappelletti94 Details may matter. This is such simple code that it should be possible to look at the assembly without going blind. If you give a sketch of the function...