blazingsql icon indicating copy to clipboard operation
blazingsql copied to clipboard

Binned pinned buffer pool allocation for communication

Open wmalpica opened this issue 3 years ago • 0 comments

In the new communication we use a pinned memory allocator that is allocated in chunks. If the chunk size is defined to be small, then for small messages there is little overhead, but for large messages, then we need a lot of chunks, which is inefficient.

We want to implement a pinned memory allocator that can allocate in different sized chunks, perhaps just a small and large size or perhaps a couple more sizes, but with the idea that we can give small chunks to small messages and large chunks to large messages.

wmalpica avatar Feb 05 '21 15:02 wmalpica