glommio
glommio copied to clipboard
DataDog
Support for Zero Copy via the SENDZC family of operations
io_ring supports the registration and re-use of buffers via the SENDZC family of operations:
See: https://lwn.net/ml/linux-kernel/[email protected]/ https://lwn.net/ml/linux-kernel/[email protected]/ https://lwn.net/Articles/879724/
I'm still learning Glommio, but it seems like it'd be a fair amount of work to support:
- The creation/management of numbered contexts. Presumably these would correspond 1:1 with executor threads, but maybe not.
- An API to allocate buffers and hand them to the context.
- Additional options for sockets to be able to leverage the zero-copy buffers.
- Functionality to "collect" the free-buffers and present to the user for re-use.
- Probably lots of other stuff
If there is interest in such features, I can take a stab at implementing them. This seems like it'd be very beneficial for my use-case.
@rrichardson yes, this feature is relatively recent on io_uring so we haven't considered it yet. But it's a very good addition!
Another thing that would be fantastic would be something like this: https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-flags2&id=ef2c2c975a921046e081a6aa29732c5c7b09cdb2
I talked to Jens about it on twitter a while ago, but I don't know if that is merged yet.
For networking in general, the biggest hindrance to maximum performance is the fact that we're following the standard rust traits, that kind of force a copy. I think we'd have to have specialized APIs to really take advantage of it.
Another benefit of not needing Send/Sync is that it should be easier to manage && data strutures.
We will probably not escape some unsafe while convincing rustc about the ownership of these buffers that are co-owned by io_uring.
Hey @rrichardson - I definitely want to implement this at some point. That said, it is VERY new and there might be a few other features that we want to implement before this.. I am trying to consider adding them based on compatible version so that we aren't making a feature that only 5 guys running 'master' can even run..
We have a decent amount of experience sharing buffers with io_uring so we are definitely aware of what it will require. It's really just a matter of finding the time and someone motivated to do it. I want to do a few of these things but I haven't had the time since the birth of my first child..
We always appreciate patches from first time contributors! just throwing that out there ;-p .. anyways thanks for mentioning this and I'll keep the ticket open until we have some more traction on this specific feature...
@bryandmc congrats on your first kid!! Totally unrelated to this ticket of course, but my third is coming in a couple of months. The second was already a crazy ride, and the third a total surprise. So definitely understand you!