FastPFor
FastPFor copied to clipboard
lemire
Is the compressed data portable?
Hello authors,
This is a great library!
I have a question: Is the compressed data portable, i.e. is it dependent on specific endianess, instruction set(cpu architecture) or anything else? If it is not portable, is there such a compression library that produces portable compressed data? If so, please give me a pointer.
Thank you!
There are many choices... but it is hard to answer in the abstract, not knowing what you are looking for...
- Roaring Bitmaps have total interoperability between C, Java, Go... there is a standard format that is the same across all platforms.
- streamvbyte supports both ARM and x64 (Haswell or better).
And so forth. If you consult our README, you will notice that we reference many other libraries.
Regarding endianess... do you program for any processor that does not support little endian? I haven't seen such a processor in well over a decade.
Hi Daniel,
Thanks for your reply. The endianess question is a general one and yes in practice little endianess has prevailed, and I am targetting commercial x86 and arm architecture.
On Wed, Apr 17, 2019 at 8:33 PM Daniel Lemire [email protected] wrote:
There are many choices... but it is hard to answer in the abstract, not knowing what you are looking for...
- Roaring Bitmaps https://roaringbitmap.org are total interoperability between C, Java, Go... there is a standard format that is the same across all platforms.
- streamvbyte https://github.com/lemire/streamvbyte supports both ARM and x64 (Haswell or better).
And so forth. If you consult our README, you will notice that we reference many other libraries.
Regarding endianess... do you program for any processor that does not support little endian? I haven't seen such a processor in well over a decade.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lemire/FastPFor/issues/50#issuecomment-484063361, or mute the thread https://github.com/notifications/unsubscribe-auth/AEVdkZBxLOvDXPXCK_vBwaF7MKObks7Nks5vhxSpgaJpZM4c0fWq .
Depending on what you are after, either streamvbyte or Roaring Bitmaps might fit the bill. Or not... it depends of your application.