bmv2 low throughput while using physical interfaces

[shahid2636]

Hi, I have installed the switch using as follows ./autogen.sh && ./configure --disable-logging-macros --disable-elogger 'CFLAGS=-O3' 'CXXFLAGS=-O3'

I performed the stress test sudo python stress_test_ipv4.py and the results are as follows

Running iperf measurement 1 of 5 912 Mbps Running iperf measurement 2 of 5 920 Mbps Running iperf measurement 3 of 5 905 Mbps Running iperf measurement 4 of 5 944 Mbps Running iperf measurement 5 of 5 917 Mbps Median throughput is 917 Mbps

system configurations are as follows. Processor : Intel® Xeon(R) CPU E5-2620 v4 @ 2.10GHz × 16 Memory: 16G 2 USB 1G interfaces.

I started Switch in the following way sudo simple_switch -i 3@enx00e04c680065 -i 4@enx00e04c680048 ho.json

my P4 program only have 3 tables. when I try to run iperf from systems connected to port 3 and 4. I only see the bandwidth around 1 Mbits/sec. I have no idea why is this happening.

Any suggestions?

Originally posted by @shahid2636 in https://github.com/p4lang/behavioral-model/issues/537#issuecomment-652871982

[antoninbas]

You may want to capture the traffic with wireshark and check if there is anything strange. It could be a checksum or fragmentation issue.

[shahid2636]

Hi, I have captured traffic using wireshark on all the interfaces. IperfClient:eth0 bmv2:3 bmv2:4 iperfServer:eth0

connection is as following between nodes. IperfClient:eth0 <-> bmv2:3(switch)bmv2:4 <->iperfServer:eth0

I see the packet size is strange at bmv2:3. packets captured at IperfClient:eth0 are all the of size 1514 while at bmv2:3 the size of captured packets varies as 7306,1514,2962 during the life time of connection. Only packets with size 1514 bytes exit bmv2 rest of all are dropped. Any suggestions?

[jafingerhut]

Very odd. Have you captured and looked at the larger packets, and compared their headers and payload contents against the 1514-byte packets that you expect to see?

[shahid2636]

Yes I did. Header fields like MAC, IP and port numbers were same. checksum was recalculated and Data was combined together. I gave it a google search and found out that the receiving interface was combining the packets. They call it large receive offload(LRO) and definition can be found here on Mellanox website there is a blog which describes how to turn LRO on or off in kernel here.

[shahid2636]

I successfully turned off GRO instead of LRO. LRO was already set to off [fixed]. turning GRO off solved the packet size issue and I can 1514 bytes packets are going through. All the checksum (ip,tcp) are correct when packets come out of bmv2.

throughput is improved to around 70 Mbits/sec which is quite smaller than I am expecting. I need at least 200 Mbps to carry out my experiments.

any suggestions will be appreciated.

[antoninbas]

Did you follow all the suggestions in https://github.com/p4lang/behavioral-model/blob/master/docs/performance.md?

[github-actions[bot]]

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment, or this will be closed in 180 days