enet icon indicating copy to clipboard operation
enet copied to clipboard

sendmmsg & Generic Segmentation Offloading?

Open CreativePSofficial opened this issue 2 years ago • 4 comments

Hello,

I wanted to ask you @lsalzman, whether ENet could see an optimization with GSO and sendmmsg (instead of the plain 'sendmsg') in order to optimize throughput? ( see: https://blog.cloudflare.com/accelerating-udp-packet-transmission-for-quic/ ) I have heard QUIC already leverages these, so I am asking how much sense it would make to implement them in ENet at least on the unix/linux side of things...

Thank you!

CreativePSofficial avatar Feb 06 '23 07:02 CreativePSofficial

@lsalzman Hello,

are there any updates on this?

CreativePSofficial avatar Mar 10 '23 21:03 CreativePSofficial

Hi, sendmmsg optimizes CPU by reducing system calls. It should be relatively easy to implement by accumulating mmsghdr inside enet_socket_send and flushing it on buffers' limit reach or by direct calls of enet_socket_send_flush. ENetSocket can be extended to a structure to store intermediate buffers. BTW, a similar technique can be used for receiving, with recvmmsg.

dimage1 avatar Mar 21 '24 08:03 dimage1

Hello sir, thanks for your reply.

Yes... We have sendmmsg in already, but now I am asking about GSO, how could this be possibly done?

I am looking forward to hearing from you. Thank you.

CreativePSofficial avatar Mar 21 '24 09:03 CreativePSofficial

Sorry, I don't know a good way to benefit from GSO on enet. Enet already tries to split data close to the MTU size. All enet packets are below MTU, but have various sizes. So pushing them to the kernel as one big buffer (e.g. 64K) with predefined segment size might break enet.  A complete OS kernel bypassing can be another way to go. Technologies like DPDK will help there.

dimage1 avatar Mar 21 '24 15:03 dimage1