emmericp
Measuring latency when switch tags a new IPOption header
I am trying to measure latency for my routing algorithm which: appends an IPOption header to an incoming packet and forwards the packet.
I am using Intel Ethernet Server Adapter X520-DA2.
As soon as I run the l3-load-latency.lua script, I get zero latency results (last line in output):
[INFO] Initializing DPDK. This will take a few seconds...
EAL: Detected 8 lcore(s)
EAL: Probing VFIO support...
EAL: PCI device 0000:01:00.0 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 8086:10fb net_ixgbe
EAL: PCI device 0000:01:00.1 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 8086:10fb net_ixgbe
[INFO] Found 2 usable devices:
Device 0: 00:1B:21:BC:E1:2E (Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection)
Device 1: 00:1B:21:BC:E1:2F (Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection)
PMD: ixgbe_dev_link_status_print(): Port 0: Link Up - speed 0 Mbps - half-duplex
PMD: ixgbe_dev_link_status_print(): Port 1: Link Up - speed 0 Mbps - half-duplex
[INFO] Waiting for devices to come up...
[INFO] Device 1 (00:1B:21:BC:E1:2F) is up: 10000 MBit/s
[INFO] Device 0 (00:1B:21:BC:E1:2E) is up: 10000 MBit/s
[INFO] 2 devices are up.
[INFO] Performing ARP lookup on 4.1.1.1
[INFO] Performing ARP lookup on 4.1.1.1
[INFO] ARP lookup failed, using default destination mac address
[INFO] ARP lookup failed, using default destination mac address
[WARN] Packet size 60 is smaller than minimum timestamp size 84. Timestamped packets will be larger than load packets.
[Device: id=0] TX: 14.88 Mpps, 7618 Mbit/s (9999 Mbit/s with framing)
[Device: id=1] RX: 13.43 Mpps, 7735 Mbit/s (9884 Mbit/s with framing)
[Device: id=0] TX: 9.62 Mpps, 4926 Mbit/s (6466 Mbit/s with framing)
[Device: id=1] RX: 9.70 Mpps, 5590 Mbit/s (7143 Mbit/s with framing)
[Device: id=0] TX: 9.74 Mpps, 4987 Mbit/s (6546 Mbit/s with framing)
[Device: id=1] RX: 9.70 Mpps, 5589 Mbit/s (7141 Mbit/s with framing)
[Device: id=0] TX: 9.77 Mpps, 5002 Mbit/s (6565 Mbit/s with framing)
[Device: id=1] RX: 9.74 Mpps, 5612 Mbit/s (7171 Mbit/s with framing)
[Device: id=0] TX: 9.85 Mpps, 5043 Mbit/s (6618 Mbit/s with framing)
[Device: id=1] RX: 9.80 Mpps, 5646 Mbit/s (7214 Mbit/s with framing)
^C[Device: id=0] TX: 9.75 (StdDev 0.09) Mpps, 4990 (StdDev 48) Mbit/s (6549 Mbit/s with framing), total 58939510 packets with 3772135576 bytes (incl. CRC)
[Device: id=1] RX: 9.74 (StdDev 0.05) Mpps, 5609 (StdDev 27) Mbit/s (7167 Mbit/s with framing), total 57488148 packets with 4139153592 bytes (incl. CRC)
Samples: 0, Average: nan ns, StdDev: 0.0 ns, Quartiles: nan/nan/nan ns
I am able to obtain latency results for the normal IP Forwarding scenario where no IPOption header is added. What changes should I make in the Lua scripts to obtain proper latency measurements?
Might be the flex filter offset, can you try to modify the offset in the device configuration here? https://github.com/libmoon/libmoon/blob/606ccdae62a4a2104ad8fc7225407f08235bf309/src/device.c#L93
But no idea if the 82599 supports IP options in the PTP stack (it should, technically)
My IPOption header is of 8 bytes, hence I made the below-mentioned changes. It didn't give the required results.
Original:
.src_offset = { 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57 },
After changes:
.src_offset = { 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65 },
I am not sure if I have done it correctly. Being very new to DPDK and MoonGen, still I have not understood what exactly this offset is trying to do.
I did search for it online but am not able to find much useful explanation.
Any help in this regard will be very useful for me. Thanks.
I think the offset should be set as
.src_offset = { 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65 },
for my use case.
It still gives me zero-latency measurements.
Is there any other code line that I need to change?