nff-go icon indicating copy to clipboard operation
nff-go copied to clipboard

Cannot init port when running with 2 DPDK port

Open ntcong opened this issue 4 years ago • 3 comments

I have 2 port using DPDK driver

python $NFF_GO/dpdk/dpdk/usertools/dpdk-devbind.py --status

Network devices using DPDK-compatible driver
============================================
0000:0b:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=vmxnet3
0000:1b:00.0 'VMXNET3 Ethernet Controller 07b0' drv=igb_uio unused=vmxnet3

Network devices using kernel driver
===================================
0000:03:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens160 drv=vmxnet3 unused=igb_uio *Active*
0000:13:00.0 'VMXNET3 Ethernet Controller 07b0' if=ens224 drv=vmxnet3 unused=igb_uio

I can use each one of those port individually

sudo ./gopacketParserExample -inport 0 -outport 0
EAL: Detected 16 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:03:00.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 15ad:7b0 net_vmxnet3
EAL: PCI device 0000:0b:00.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 15ad:7b0 net_vmxnet3
EAL: PCI device 0000:13:00.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 15ad:7b0 net_vmxnet3
EAL: PCI device 0000:1b:00.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 15ad:7b0 net_vmxnet3
------------***------ Initializing scheduler -----***------------
DEBUG: Scheduler can use cores: [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14]
------------***---------- Creating ports ---------***------------
DEBUG: Port 0 MAC address: 00:50:56:90:be:28
DEBUG: Port 1 MAC address: 00:50:56:90:92:84
------------***------ Starting FlowFunctions -----***------------

But if I use 1 inport and another outport, my app will crash with "Cannot init port"

sudo ./gopacketParserExample -inport 0 -outport 1
------------***-------- Initializing DPDK --------***------------
EAL: Detected 16 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: PCI device 0000:03:00.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 15ad:7b0 net_vmxnet3
EAL: PCI device 0000:0b:00.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 15ad:7b0 net_vmxnet3
EAL: PCI device 0000:13:00.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 15ad:7b0 net_vmxnet3
EAL: PCI device 0000:1b:00.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 15ad:7b0 net_vmxnet3
------------***------ Initializing scheduler -----***------------
DEBUG: Scheduler can use cores: [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14]
------------***---------- Creating ports ---------***------------
DEBUG: Port 0 MAC address: 00:50:56:90:be:28
vmxnet3_dev_configure(): ERROR: Number of rx queues not power of 2
Port1 dev_configure = -22
ERROR: Cannot init port  1 !

ntcong avatar Aug 05 '19 10:08 ntcong

I think the problem is in internal/low/low.h

	if (willReceive) {
		rx_rings = inIndex;
	} else {
		rx_rings = 0;
	}

rx_rings is 0 and dpdk's vmxnet3 driver doesn't like that dpdk/dpdk/drivers/net/vmxnet3/vmxnet3_ethdev.c

	if (!rte_is_power_of_2(dev->data->nb_rx_queues)) {
		PMD_INIT_LOG(ERR, "ERROR: Number of rx queues not power of 2");
		return -EINVAL;
	}

ntcong avatar Aug 05 '19 10:08 ntcong

@ntcong Do you mean that when a port is set for output only, it gets rx_rings equal to zero and this causes driver to fail?

gshimansky avatar Aug 07 '19 19:08 gshimansky

yes that is correct

ntcong avatar Aug 08 '19 02:08 ntcong