Mellanox device dropping odp generated broadcast/multicast packet

[subhajitchatu]

Hi, I am facing a strange issue with odp dpdk stack .. I am running odp_generator example binary for generating packets and receiving those packets in another vm. below are my observations

1. Sending packets having destination mac Broadcast/multicast
   • working with odp_generator using virtio interface
   • NOT Working with odp_generator using mellanox connectx6Dx interface
   • Working if I don’t use odp_generator (i.e odp stack) and generate arp packets from linux using the same mellanox interface.
   • Working with dpdk testpmd sample application using mellanox interface
2. Sending packets having destination mac unicast
   • working fine with odp_generator using mellanox interface
   • Working fine with dpdk testpmd using mellanox interface
   • Working fine if not using odp stack

The packet drop is happening in mellanox hw as I can see rx counter is increasing for the interface in case of odp_generator using mellanox interface . Moreover I can see tx counter of the same interface is increasing at the same time in this particular case (broadcast/multicast). Is there any add on dpdk port configuration which we can try for mlx5 driver ? It can also be possible that there is some dpdk port or global configuration which has triggered some rte_flow rules that are dropping packets for broadcast/multicast mac … It would be helpful if I can get some pointers to debug this issue. Thanks Subhajit

[MatiasElo]

Hi,

With the default ODP configuration file (odp-linux-dpdk.conf) multicast reception should be enabled (implementation calls rte_eth_allmulticast_enable()). Could you please configure ODP with --enable-debug=full option and provide the application startup log. I'm mainly interested about the DPDK pmd drived name and options: E.g.

odp_packet_dpdk.c:198:init_options():DPDK interface (net_i40e): 0
odp_packet_dpdk.c:200:init_options():  multicast:   1
odp_packet_dpdk.c:201:init_options():  num_rx_desc: 128
odp_packet_dpdk.c:202:init_options():  num_tx_desc: 256
odp_packet_dpdk.c:203:init_options():  rx_drop_en:  0

Also, which DPDK version are you using?

You could also test enabling promisc mode in the generator application:

diff --git a/example/generator/odp_generator.c b/example/generator/odp_generator.c
index 7a070235b..807b582dd 100644
--- a/example/generator/odp_generator.c
+++ b/example/generator/odp_generator.c
@@ -652,6 +652,9 @@ static int create_pktio(const char *dev, odp_pool_t pool,
                }
        }
 
+       odp_pktio_promisc_mode_set(itf->pktio, true);
+       odp_pktio_print(itf->pktio);
+
        ret = odp_pktio_start(itf->pktio);
        if (ret)
                ODPH_ABORT("Error: unable to start %s\n", dev);

[subhajit-cdot]

Thanks for your reply. I have checked that odp application using mellanox interface is working fine in another VM having ubuntu desktop image + kernel version 5.11.0-38-generic with both dpdk-19.11 and dpdk-20.11. It is not working in the VM which is having 4.15.0-162-generic kernel + ubuntu cloud image. I will try with setting promisc mode and update here.

Thanks Subhajit

[subhajit-cdot]

It's not working even after setting promisc mode in that setup. Actually I have tested with dpdk pktgen as well. It is also not working for dmac ff:ff:ff:ff:ff:ff:ff or for any multicast dmac. Are you able to guess any other workaround or issue in this setup? Could linux headers be a problem here? The issue only comes with mellanox device, but not with virtio devices. Thanks Subhajit

[MatiasElo]

Sounds like an issue in the Mellanox drivers, so there is very little we can do. I'd verify that the latest Mellanox drivers for the correct OS are being used. Also, if I remember correctly, you can still control Mellanox interfaces from Linux, while they are used by DPDK, so you could try enabling multicast with e.g. ifconfig.