BNX2 connectivity/packet loss .. see also #292

[thctlo]

Hai,

im wondering, i did read the threath #292 im seeing the same thing here.. PowerEdge R515

NIC offloading gives here lots of problems, i've tried turning it off as shown here : https://wiki.tnonline.net/w/Blog/Disable_Network_Offloading_on_XenServer_and_XCP-ng and here https://xcp-ng.org/docs/networking.html

3 servers all 3 have the problem, all R515, all latest firmware. these servers are running XenServer as of 6.x upto 7.2 then i upgraded to XCP-NG, im now at 8.2

Anyone suggestions on how to disable nic offloading fully? HOST and VM Guests.

[olivierlambert]

Do you still have the issue after doing what's explained here? https://wiki.tnonline.net/w/Blog/Disable_Network_Offloading_on_XenServer_and_XCP-ng

edit: tso disabled should be enough.

[thctlo]

yes, sadly i do, i have 3 exact same servers.

2 have now loading disabled, 1 not.. uh... now i look again and its off on all 3 Xen Hosts. ok, so thats matter of waiting more.

but when i look again in the guest, still on, not all, but tx-checksumming still is. Is there a specific order i must follow? i have also turned it off already in a VM still on, do i need to set it everything after a reboot ?

[thctlo]

this is last output on a VM.

ethtool -k eth0
Features for eth0:
rx-checksumming: on [fixed]
tx-checksumming: on
        tx-checksum-ipv4: on [fixed]
        tx-checksum-ip-generic: off [fixed]
        tx-checksum-ipv6: on
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: on
        tx-scatter-gather: on
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp-mangleid-segmentation: off
        tx-tcp6-segmentation: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: off [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: on [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]

[olivierlambert]

It doesn't matter for a VM because it's using PV drivers. What matters is the host/Dom0.

[thctlo]

this is the output of eth0 on HOST

Features for eth0:
rx-checksumming: off
tx-checksumming: off
tx-checksum-ipv4: off
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: off
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: off
tx-tcp-segmentation: off [requested on]
tx-tcp-ecn-segmentation: off [requested on]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: off [requested on]
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]

so resume on that only these are still on the HOST.

scatter-gather: on tx-scatter-gather: on rx-vlan-offload: on tx-vlan-offload: on receive-hashing: on highdma: on [fixed]

I'll do a reboot tonight and verify it again. this problem started i "believe" after the upgrade to XCP-NG 8.x I just cant recall exacly which version.

But BIG thanks for the bazing fast reply.

[nagilum99]

Did you fully shutdown the VM and start it again? Some changes, especially to VM meta configuration, don't apply before you fully shut it down, because it's not reloaded on reboot.

[olivierlambert]

There's 2 things:

1. The xe command will apply on host boot only
2. ethtool will apply it in live, no need to reboot the host

[thctlo]

Thanks guys, i'll have to wait for the weekend to reboot my XCP-NG Serverhosts. so if i understand it right.

I apply to the host per UUID (VIF/PIF). i reboot the host servers and then i verify the guest VM's again.

(sorry english is not my native language) then the setting is also defaultl in the VM?

[olivierlambert]

That's correct. And there's nothing to do in your VMs.

[thctlo]

So i tripple checked my HOST yesterday, everyting is/was set. all that was still on was this :

ethtool --show-offload eth0|grep ": on"
generic-receive-offload: on
rx-vlan-offload: on
tx-vlan-offload: on
receive-hashing: on
highdma: on [fixed]

host is rebooted, i checked my VM again and i still see this :

rx-checksumming: on [fixed]
tx-checksumming: on
        tx-checksum-ipv4: on [fixed]
        tx-checksum-ipv6: on
scatter-gather: on
        tx-scatter-gather: on
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
tx-gso-robust: on [fixed]

why is tx-checksumming off in HOST and on in the guest.

I would have expected that it was of also now, any other suggestions? (i can make a script and turn if off on boot, but that only a temp fix. ).

[olivierlambert]

It doesn't matter on the VM. The VM isn't using any real network card (nor hardware), so consider any "hardware" information in the VM as irrelevant.

[thctlo]

ok, so, if i understand it right now, even the VM shows that, its not applied, only the HOSTS setting are whats about. And a big big thanks Oliver on these quick replies. very very apriciated.

[olivierlambert]

That's correct :)

You are welcome!