netlink
netlink copied to clipboard
netlink: add ExecuteFunc method to process large responses with low mem overhead
One of the great things you can do with netlink is query for detailed stats on all open TCP connections, including e.g. retransmits, how much data is in socket buffers, the computed BBR bandwidth estimates, etc. Unfortunately, if you ask the kernel for this info on a host with a lot of connections, the []netlink.Message
slice returned by netlink/Conn.Execute
can take 50+ MB of space, when the slice will be (likely) be thrown away after its iterated over once.
This PR adds a new method ExecuteFunc
where a user can pass in a callback function that is called once per message. Becuase of how Conn.receive
/the netlink recvmsg
API works in batches/streaming, this dramatically reduces the peak RSS/memory overhead required to process a large stream of netlink message responses.
To try to minimize duplication, I refactored receive
into receiveEach
which operates on a callback, changed lockedReceive
to call receiveEach
, and then added a new lockedReceiveEach
which is callled by ExecuteFunc
(I started with the Each
suffix, but then saw that a bunch of the callback variants in the Go stdlib have a Func
suffix. LMK if you prefer one or the other or something else and I can standardize the naming).