PcapPlusPlus icon indicating copy to clipboard operation
PcapPlusPlus copied to clipboard

DPDK uses more than 32 cores

Open gyl30 opened this issue 10 months ago • 8 comments

Question

I noticed that both CoreMask and SystemCores are only 32, can we support more?

Operating systems

No response

gyl30 avatar Feb 14 '25 05:02 gyl30

How many cores do you have? I think maybe we can make it to 64?

tigercosmos avatar Feb 14 '25 07:02 tigercosmos

How many cores do you have? I think maybe we can make it to 64?

My current project aims to use 32 cores. However, since the MasterCore occupies one core, the current implementation can only use 30 cores, as the number of usable cores needs to be an even number. More generally, when using multiple network cards, each card or queue may require a dedicated core. Therefore, I believe we should increase the number to 128 or 256 to accommodate a wider range of needs. Perhaps we can gather more opinions on this.

gyl30 avatar Feb 14 '25 08:02 gyl30

128 or 256

Having the CoreMask be >64 cores would require a change of the CoreMask from a std::uint32_t to a dedicated struct as the bitmask would be split up between 2 64-bit integers. Perhaps we can add a CoreMaskLong if we don't want to break code depending on CoreMask being integral type?

Also I am not sure if DPDK can be initialized 256 cores. The highest I saw in documentation was initializing to core index 128.

Dimi1010 avatar Feb 14 '25 08:02 Dimi1010

128 or 256

Having the CoreMask be >64 cores would require a change of the CoreMask from a std::uint32_t to a dedicated struct as the bitmask would be split up between 2 64-bit integers. Perhaps we can add a CoreMaskLong if we don't want to break code depending on CoreMask being integral type?

Also I am not sure if DPDK can be initialized 256 cores. The highest I saw in documentation was initializing to core index 128.

As you said, if we need to change the CoreMask limit, we may need to design the code properly

gyl30 avatar Feb 14 '25 09:02 gyl30

DPDK sets its core limit using the RTE_MAX_LCORE define in rte_config.h, but you can adjust it to support 512 or even 1024 cores by changing the configuration and recompiling. I suggest updating PcapPlusPlus to support an 'unlimited' coremask for more flexibility.

I can take this on, but I won't be able to start until a couple of weeks from now.

SesomB avatar Feb 14 '25 22:02 SesomB

@SesomB Yes, it will be great. Thanks.

tigercosmos avatar Feb 15 '25 07:02 tigercosmos

DPDK sets its core limit using the RTE_MAX_LCORE define in rte_config.h, but you can adjust it to support 512 or even 1024 cores by changing the configuration and recompiling. I suggest updating PcapPlusPlus to support an 'unlimited' coremask for more flexibility.

I can take this on, but I won't be able to start until a couple of weeks from now.

@SesomB I think that could be a nice improvement to this library (mostly for DPDK but not only). It'd be great if you can work on it!

seladb avatar Mar 20 '25 07:03 seladb

@SesomB are you still planning to work on this task?

seladb avatar May 01 '25 09:05 seladb