blazingsql
blazingsql copied to clipboard
Binned pinned buffer pool allocation for communication
In the new communication we use a pinned memory allocator that is allocated in chunks. If the chunk size is defined to be small, then for small messages there is little overhead, but for large messages, then we need a lot of chunks, which is inefficient.
We want to implement a pinned memory allocator that can allocate in different sized chunks, perhaps just a small and large size or perhaps a couple more sizes, but with the idea that we can give small chunks to small messages and large chunks to large messages.