Firecracker networking improvements

[bchalios]

This is a tracker for investigating improvements in the Firecracker networking stack. We are mainly looking to improve performance (throughput, latency and CPU utilization) of the emulation logic. However, we want to keep an eye on simplifying the control plane.

Currently, we want to focus our efforts towards a vhost-user-net data-plane, however, we will evaluate other alternatives along the way.

Additional context

Related issues:

Macvtap: #1933 vhost-net: #3707, #4312

Checks

• [x] Have you searched the Firecracker Issues database for similar requests?
• [x] Have you read all the existing relevant Firecracker documentation?
• [x] Have you read and understood Firecracker's core tenets?

[sagoresarker]

I think it can extend its functionality by not relying solely on TUN/TAP.

[DemiMarie]

I think the most important change here is to move away from the Linux tap interface to something with much better performance. Tap devices cannot support a large number of packets per second, whereas vhost-net, XDP, and other solutions can.