bitsery
bitsery copied to clipboard
fraillt
io_uring based stream adapter
not sure if you've heard about io_uring on Linux but it basically allows for syscall-less asynchronous operations and IO through the kernel. It's incredibly fast. I can show you a generic read/write file object i wrote to do DirectIO.
anyway this would obviously be the most performant way to stream to/from disk.
not sure how you'd feel about including a new stream adapter into the library that had a dependency on https://github.com/axboe/liburing + Linux.
even if not in the library, i assume if I wrote a new BasicBufferedOutputStreamAdapter say IOuringBufferedOutputStreamAdapter, and the same for input, and mimicked all the function calls etc, that it would "just work"?
I have a situation where I'm serializing very large (possible into the GBs) objects then writing them to disk, and don't want to blow up memory, thrash the page cache (hence DirectIO), nor waste precious seconds getting the data onto disk.
I wouldn't want to include any additional dependency. However, it would be super cool to have separate bitsery_uiring library using modern cmake and I would include a link in the readme to it, and everyone would be happy:)
If you don't mind I would like to create it, and make you collaborator (meaning you would have write access to it).
What do you say?:)
I will create a repo and will provide a cmake setup so you could start working on implementation
@victorstewart Are you still interested in this? I want to remind you, that I have created a repository to get started in here.
sorry yes, unfortunately right now i'm buried in a bunch of work for a while so i don't know when i could commit to writing it just now.