Zero-copy in the network stack

[IanSeyler]

trafficstars

http://en.wikipedia.org/wiki/Zero-copy

[hidnplayr]

Go for it! :)

[benaryorg]

So, you want a syscall (?) to send a memory range directly onto the network?

[IanSeyler]

Part of this has been implemented. os_ethernet_tx forwards the memory location of the packet directly to the network driver and the hardware handles it. No time is spent by the CPU to copy anything.

os_ethernet_rx does one copy (from the OS packet buffer to the requested memory address) so that will need to be adjusted.

[benaryorg]

It is a buffer, it will be hard to impossible to remove that, unless you let the application specify its own.

[IanSeyler]

Yep, the thought was to add an option to specify the address - sort of like adding the network callback address.

[benaryorg]

Is the buffer used for more than one packet?

[scherrey]

Are we talking about on-nic memory? Does the driver support this? If so how much is available?

I did zero copy security systems in promiscuous mode back in my security days. The cards we used had significant onboard memory which we reviewed in place and decided which data to copy into machine shared memory and which to ignore.

The call back mechanism that exists should work well for such a design if the memory is as fast or faster than system RAM otherwise it's best just to copy to system shared memory. Just need to ensure that your process talking to the nic always has a cpu assigned to it.

• - Ben On Jul 13, 2015 2:49 PM, "Katze" [email protected] wrote:

Is the buffer used for more than one packet?

— Reply to this email directly or view it on GitHub https://github.com/ReturnInfinity/BareMetal-OS/issues/43#issuecomment-120841007 .

[IanSeyler]

That buffer, os_EthernetBuffer, only stores the most recent packet that was received by hardware. It used to be a ring buffer but that was removed once the callback code was enabled.

We're talking about system memory here and the current drivers do not support on-nic memory.

Heres what is currently happening when a packet comes it:

1. Hardware grabs the packet and stores it in a system buffer.
2. App calls os_ethernet_rx to copy the packet in the system buffer to its own storage location.
3. Repeat

Here is what we want for zero-copy:

1. Hardware grabs the packet and stores it in a buffer specified by the application.
2. Repeat.

[benaryorg]

I think it would be an overall good solution if the application could specify it's own handlers for all interrupts (without overwriting the system ones).

That would solve many problems. It would enable writing a task-switching mechanism and would enable some better mechanisms than polling.

We also had exception handling then.

[IanSeyler]

At the moment the only two system interrupts are caused by the RTC and the network. Each of those supports an application defined callback so polling isn't required.

[benaryorg]

Sorry, but I haven't had time yet to have a deeper look into BareMetalOS.

Isn't there a Zero-Copy network "stack" then?

[IanSeyler]

At the moment BareMetal does not have zero-copy in its network stack.

Also I took a look at the existing network drivers and they contain the dredded rep movsb instruction. This change will involve driver modification as well. This is probably left overs from the ring buffer system that was removed.