IPv6 packet tranmission seems orders of magnitude slower than IPv4

[theleos88]

Hello, I was trying to use MoonGen to generate IPv6 packets. I manage to achieve line rate with the example l3-load-latency using 2 cores (64 byte pkts,14.8 Mpps).

I just modified the example to create IPv6 packets (all calls are modified accordingly).

local function fillUdp6Packet(buf, len)
    buf:getUdp6Packet():fill{
        ethSrc = queue,
        ethDst = DST_MAC,
        ip6Src = "1111::1",
        ip6Dst = "1221::1",
        udpSrc = SRC_PORT,
        udpDst = DST_PORT,
        pktLength = len
    }
end

With this configuration, I achieve 1 Gbps using 15 cores.

[Device: id=1] TX: 2.12 Mpps, 1088 Mbit/s (1427 Mbit/s with framing)
[Device: id=0] RX: 2.17 Mpps, 1113 Mbit/s (1461 Mbit/s with framing)

Is IPv6 generation this slow? Or should I use it in another way? Thank you.

[theleos88]

Hello, I performed some more experiments. The issue is with the modification of the flows at per packet level, so basically, in the slave:

    while mg.running() do
        b = bufs:alloc(size)
        for i, buf in ipairs(bufs) do
            local pkt = buf:getUdp6Packet()
            pkt.ip6.dst:set(baseIP6 + counter)  -- This is really, really slow 
            counter = incAndWrap(counter, flows)
        end

I dissected the bottleneck, and verified three scenarios:

• remove the pkt.ip6.dst.set (sending continuously the same packet)
• using an external C function to set the IPv6 address (per packet)
• external C function per batch

I attach here the results for a single-slave scenario

| No dst set -------  14.88 Mpps Tx|
| lua pkt.ip6 -------  0.85   Mpps Tx|
| clib.increment --- 14.88 Mpps Tx|
| clib.random ------ 8.44   Mpps Tx|
| clib.rand_batch - 12.92 Mpps Tx|
| clib.round_rob --- 14.88 Mpps Tx|

The outcome is that pkt.ip6.set + counter is not working at line rate, but this is strange to me because it is the same thing used for ipv4 in the example load_latency which was working at line rate.

The external c code is written using the c-integration example. The function just parses an rte_mbuf and modify the ipv6 destination address. I replaced the set function with the clib.fill_single_ip6(buf) corresponding to a C function that is as simple as:

void fill_single_ip6(struct rte_mbuf* mbuf) {
    /* Handle IPv6 headers.*/
        uint8_t *pkt = rte_pktmbuf_mtod(mbuf, uint8_t*);
        pkt+= (sizeof(struct ether_hdr));
        struct ipv6_hdr *ipv6 = (struct ipv6_hdr *)(pkt);
        ipv6->dst_addr[15] = (rand()%14)+1;
}

Hope that it helps you, anyway I'm just using C-binding to generate my ipv6 traffic.

[emmericp]

Thanks for investigating this, looks like a problem with the JIT compiler not optimizing the wrapper object for the IP address struct. Seems to be a problem with newer versions since it wasn't that slow in the past.

I'll investigate

[kthkaya]

Hi @emmericp , I am planning to use Moongen for my research involving IPv6, was wondering if there is an ETA for a fix on this issue?

[ThalabathyVenkatesan]

Hi @theleos88 , I am planning to do test for ipv6, is it possible you to share scripts that you wrote for ipv6 .

Thanks in advance.

[ThalabathyVenkatesan]

Hi @emmericp , While cloning the repo , ipv6 related scripts and files are missing. Could yo please do the needful on the same. Thanks in advance. -Thalabathy

[theleos88]

Hi all,

for my experiments I modified the built-in example for clib integration in lua. I attach in the zip archive:

• the ip6_handler directory, with the fill-ip6.c that I used to create custom IPv6 packets
• the lua script l3ip6.lua used for MoonGen to generate my IPv6 packets.

In the lua scripts there are the clib calls (commented) that you can reuse to replace the basic udp fill of the original script. Of course, this was very related to my needs, so you will need to adjust it according to your goals. Hope it helps.

ip6_clib.zip

[ThalabathyVenkatesan]

@theleos88 Thanks