lagopus cannot start normally

Open monstery opened this issue 11 years ago • 1 comments

Hi, We have two CPU Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz on socket0 and socket1(total 24 cores), memory size is 48G and 28 Intel NICs on board, but we can not initiate lagopus.

The command is only to start 24 NICs but at last it will fail. sudo /usr/sbin/lagopus -d -- -cfff -n 4 -- -pffffff

The following is our log.

user@debian:~$ sudo /usr/sbin/lagopus -d -- -cfff -n 4 -- -pffffff
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 0 on socket 1
EAL: Detected lcore 7 as core 1 on socket 1
EAL: Detected lcore 8 as core 2 on socket 1
EAL: Detected lcore 9 as core 3 on socket 1
EAL: Detected lcore 10 as core 4 on socket 1
EAL: Detected lcore 11 as core 5 on socket 1
EAL: Detected lcore 12 as core 0 on socket 0
EAL: Detected lcore 13 as core 1 on socket 0
EAL: Detected lcore 14 as core 2 on socket 0
EAL: Detected lcore 15 as core 3 on socket 0
EAL: Detected lcore 16 as core 4 on socket 0
EAL: Detected lcore 17 as core 5 on socket 0
EAL: Detected lcore 18 as core 0 on socket 1
EAL: Detected lcore 19 as core 1 on socket 1
EAL: Detected lcore 20 as core 2 on socket 1
EAL: Detected lcore 21 as core 3 on socket 1
EAL: Detected lcore 22 as core 4 on socket 1
EAL: Detected lcore 23 as core 5 on socket 1
EAL: Support maximum 64 logical core(s) by configuration.
EAL: Detected 24 lcore(s)
EAL: Setting up memory...
EAL: Ask a virtual area of 0x7b400000 bytes
EAL: Virtual area found at 0x7f7cf1a00000 (size = 0x7b400000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7f7cf1400000 (size = 0x400000)
EAL: Ask a virtual area of 0x2000000 bytes
EAL: Virtual area found at 0x7f7cef200000 (size = 0x2000000)
EAL: Ask a virtual area of 0xc00000 bytes
EAL: Virtual area found at 0x7f7cee400000 (size = 0xc00000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f7cee000000 (size = 0x200000)
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7f7ced600000 (size = 0x800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f7ced200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f7cece00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f7ceca00000 (size = 0x200000)
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7f7cec000000 (size = 0x800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f7cebc00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f7ceb800000 (size = 0x200000)
EAL: Ask a virtual area of 0x7f000000 bytes
EAL: Virtual area found at 0x7f7c6c600000 (size = 0x7f000000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f7c6c200000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7f7c6bc00000 (size = 0x400000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7f7c6b600000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f7c6b200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f7c6ae00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f7c6aa00000 (size = 0x200000)
EAL: Requesting 1024 pages of size 2MB from socket 0
EAL: Requesting 1024 pages of size 2MB from socket 1
EAL: TSC frequency is ~2000000 KHz
EAL: Master core 0 is ready (tid=71ac3800)
EAL: Core 7 is ready (tid=671e1700)
EAL: Core 8 is ready (tid=669e0700)
EAL: Core 9 is ready (tid=661df700)
EAL: Core 10 is ready (tid=659de700)
EAL: Core 11 is ready (tid=651dd700)
EAL: Core 6 is ready (tid=679e2700)
EAL: Core 5 is ready (tid=681e3700)
EAL: Core 4 is ready (tid=689e4700)
EAL: Core 3 is ready (tid=691e5700)
EAL: Core 2 is ready (tid=699e6700)
EAL: Core 1 is ready (tid=6a1e7700)
EAL: PCI device 0000:05:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d71a6d000
EAL:   PCI memory mapped at 0x7f7d71a69000
EAL: PCI device 0000:05:00.1 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d71a49000
EAL:   PCI memory mapped at 0x7f7d71a45000
EAL: PCI device 0000:05:00.2 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d71a25000
EAL:   PCI memory mapped at 0x7f7d71a21000
EAL: PCI device 0000:05:00.3 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d71a01000
EAL:   PCI memory mapped at 0x7f7d719fd000
EAL: PCI device 0000:07:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d719dd000
EAL:   PCI memory mapped at 0x7f7d719d9000
EAL: PCI device 0000:07:00.1 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d719b9000
EAL:   PCI memory mapped at 0x7f7d719b5000
EAL: PCI device 0000:07:00.2 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d71995000
EAL:   PCI memory mapped at 0x7f7d71991000
EAL: PCI device 0000:07:00.3 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d71971000
EAL:   PCI memory mapped at 0x7f7d7196d000
EAL: PCI device 0000:0c:00.0 on NUMA socket 0
EAL:   probe driver: 8086:10d3 rte_em_pmd
EAL:   PCI memory mapped at 0x7f7d7194d000
EAL:   PCI memory mapped at 0x7f7d71949000
EAL: PCI device 0000:0d:00.0 on NUMA socket 0
EAL:   probe driver: 8086:10d3 rte_em_pmd
EAL:   PCI memory mapped at 0x7f7d71929000
EAL:   PCI memory mapped at 0x7f7d71925000
EAL: PCI device 0000:83:00.0 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6cfa0000
EAL:   PCI memory mapped at 0x7f7d71b0e000
EAL: PCI device 0000:83:00.1 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6cf80000
EAL:   PCI memory mapped at 0x7f7d6cf7c000
EAL: PCI device 0000:83:00.2 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6cf5c000
EAL:   PCI memory mapped at 0x7f7d6cf58000
EAL: PCI device 0000:83:00.3 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6cf38000
EAL:   PCI memory mapped at 0x7f7d6cf34000
EAL: PCI device 0000:85:00.0 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6cf14000
EAL:   PCI memory mapped at 0x7f7d6cf10000
EAL: PCI device 0000:85:00.1 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6cef0000
EAL:   PCI memory mapped at 0x7f7d6ceec000
EAL: PCI device 0000:85:00.2 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6cecc000
EAL:   PCI memory mapped at 0x7f7d6cec8000
EAL: PCI device 0000:85:00.3 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6cea8000
EAL:   PCI memory mapped at 0x7f7d6cea4000
EAL: PCI device 0000:87:00.0 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6ce84000
EAL:   PCI memory mapped at 0x7f7d6ce80000
EAL: PCI device 0000:87:00.1 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6ce60000
EAL:   PCI memory mapped at 0x7f7d6ce5c000
EAL: PCI device 0000:87:00.2 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6ce3c000
EAL:   PCI memory mapped at 0x7f7d6ce38000
EAL: PCI device 0000:87:00.3 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7d6ce18000
EAL:   PCI memory mapped at 0x7f7d6ce14000
EAL: PCI device 0000:89:00.0 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7cf19e0000
EAL:   PCI memory mapped at 0x7f7cf19dc000
EAL: PCI device 0000:89:00.1 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   PCI memory mapped at 0x7f7cf19bc000
EAL:   PCI memory mapped at 0x7f7cf19b8000
EAL: PCI device 0000:89:00.2 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   0000:89:00.2 not managed by UIO driver, skipping
EAL: PCI device 0000:89:00.3 on NUMA socket 1
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL:   0000:89:00.3 not managed by UIO driver, skipping
PANIC in app_init_rings_tx():
Algorithmic error (no I/O core to handle TX of port 16)
12: [/usr/sbin/lagopus() [0x40b865]]
11: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f7d6ece8ead]]
10: [/usr/sbin/lagopus() [0x40c79c]]
9: [/usr/sbin/lagopus() [0x40c154]]
8: [/usr/lib/liblagopus_util.so.0(lagopus_module_initialize_all+0x76) [0x7f7d6f91ff19]]
7: [/usr/lib/liblagopus_util.so.0(+0x1f986) [0x7f7d6f91f986]]
6: [/usr/lib/liblagopus_dataplane.so.0(datapath_initialize+0x41) [0x7f7d700cd555]]
5: [/usr/lib/liblagopus_dataplane.so.0(lagopus_datapath_init+0xc5) [0x7f7d700f7ba5]]
4: [/usr/lib/liblagopus_dataplane.so.0(app_init+0x18) [0x7f7d700f78c1]]
3: [/usr/lib/liblagopus_dataplane.so.0(app_init_rings_tx+0xe5) [0x7f7d701010a2]]
2: [/usr/sbin/lagopus(__rte_panic+0xc3) [0x40b6f5]]
1: [/usr/sbin/lagopus() [0x4a2463]]
user@debian:~$

Oct 21 '14 02:10 monstery

Thank you for your report. Due to he current limitation of DPDK and lagopus, the maximum number of ports is limited to 16 in Lagopus and 32 in DPDK.

Please change the source code as follows:

src/dataplane/dpdk/dpdk.h
 @@ -70,11 +70,11 @@
 #endif

 #ifndef APP_MAX_NIC_RX_QUEUES_PER_IO_LCORE
-#define APP_MAX_NIC_RX_QUEUES_PER_IO_LCORE 16
+#define APP_MAX_NIC_RX_QUEUES_PER_IO_LCORE 32
 #endif

 #ifndef APP_MAX_NIC_TX_PORTS_PER_IO_LCORE
-#define APP_MAX_NIC_TX_PORTS_PER_IO_LCORE 16
+#define APP_MAX_NIC_TX_PORTS_PER_IO_LCORE 32
 #endif
 #if (APP_MAX_NIC_TX_PORTS_PER_IO_LCORE > APP_MAX_NIC_PORTS)
 #error "APP_MAX_NIC_TX_PORTS_PER_IO_LCORE too big"

Dec 16 '14 01:12 ynkjm