RVVM icon indicating copy to clipboard operation
RVVM copied to clipboard

tap_linux: Bad checksums with RTL8169, broken by default

Open LekKit opened this issue 2 years ago • 1 comments

The issue

  • Newer emulated RTL8169 NIC doesn't provide working network with tap_linux backend and a Linux guest

Problem investigation

  • Older OpenCores Ethernet NIC didn't support checksum offload; The guest would compute proper TCP/IP checksum in software
  • Newer (and default) Realtek RTL8169 NIC supports checksum offload, which is turned on by default; The guest (rightfully so) doesn't compute the checksum in transmitted packets and just gives them to NIC like that
  • The RVVM TAP API receives the TX packet, at which point it's either parsed by tap_user which doesn't care about checksums, or it is passed to TUN/TAP Linux interface via tap_linux
  • Finally, the TUN/TAP Linux interface doesn't have means of "checksum offload"; It simply drops those packets since the checksum is bad and leaves no option to compute the checksum by the kernel, or better offload it to the host NIC and omit any overhead from software checksumming at all

Workarounds

  • Disable TX checksum offload in the guest via ethtool -K enp1s0 tx off

Possible proper fixes

  • Find a way to ask the host TUN/TAP device for TX checksum offload; No piece of documentation has a clue on that, it could be simply a limitation on Linux side...
  • Implement software TX checksumming in tap_linux implementation; Slightly complicated and carries some overhead

If anyone has an advice on proper TUN/TAP TX checksum offload, it would be nice to point out here. Otherwise it should fallback to (arguably, more ugly) approach of working around the system interface limitations, or simply deprecate the TUN/TAP backend when tap_user is at more usable state.

LekKit avatar Mar 25 '23 12:03 LekKit

Trying out ioctl(tap->fd, TUNSETOFFLOAD, TUN_F_CSUM | TUN_F_TSO4 | TUN_F_TSO6 | TUN_F_TSO_ECN | TUN_F_UFO) doesn't seem to work, even tho TUN_F_CSUM implies "You can hand me unchecksummed packets."...

IFF_VNET_HDR might be the thing we are looking after, allowing to pass "GSO and checksum information", but that'd take a bit more effort to implement

LekKit avatar Mar 27 '23 00:03 LekKit