RVVM
RVVM copied to clipboard
tap_linux: Bad checksums with RTL8169, broken by default
The issue
- Newer emulated RTL8169 NIC doesn't provide working network with
tap_linuxbackend and a Linux guest
Problem investigation
- Older OpenCores Ethernet NIC didn't support checksum offload; The guest would compute proper TCP/IP checksum in software
- Newer (and default) Realtek RTL8169 NIC supports checksum offload, which is turned on by default; The guest (rightfully so) doesn't compute the checksum in transmitted packets and just gives them to NIC like that
- The RVVM TAP API receives the TX packet, at which point it's either parsed by
tap_userwhich doesn't care about checksums, or it is passed to TUN/TAP Linux interface viatap_linux - Finally, the TUN/TAP Linux interface doesn't have means of "checksum offload"; It simply drops those packets since the checksum is bad and leaves no option to compute the checksum by the kernel, or better offload it to the host NIC and omit any overhead from software checksumming at all
Workarounds
- Disable TX checksum offload in the guest via
ethtool -K enp1s0 tx off
Possible proper fixes
- Find a way to ask the host TUN/TAP device for TX checksum offload; No piece of documentation has a clue on that, it could be simply a limitation on Linux side...
- Implement software TX checksumming in
tap_linuximplementation; Slightly complicated and carries some overhead
If anyone has an advice on proper TUN/TAP TX checksum offload, it would be nice to point out here. Otherwise it should fallback to (arguably, more ugly) approach of working around the system interface limitations, or simply deprecate the TUN/TAP backend when tap_user is at more usable state.
Trying out ioctl(tap->fd, TUNSETOFFLOAD, TUN_F_CSUM | TUN_F_TSO4 | TUN_F_TSO6 | TUN_F_TSO_ECN | TUN_F_UFO) doesn't seem to work, even tho TUN_F_CSUM implies "You can hand me unchecksummed packets."...
IFF_VNET_HDR might be the thing we are looking after, allowing to pass "GSO and checksum information", but that'd take a bit more effort to implement