Perftest tools use shared CQ for mutiple QPs

[bharatpotnuri]

Hi, I see that with multiple QP support to perftest (-q), single uber CQ is created and shared by all the QPs in an instance (ib_read_lat/ib_write_bw etc.). If more QPs are used per instance say -q 1000, CQ becomes too big and run out of max device supported cqe per CQ and may fail creating CQ or fail polling the CQ with limited size (device supported max size) for more number QPs.

Any reason behind using single large CQ for multiple QPs? Please let me know if creating CQ per QP makes sense.

Thanks

[sshaulnv]

I tried to reproduce the problem with a larger number of QPs, and there was no problem. Can you provide the exact command you used? also, it will be helpful to get more information about your setup and chip , driver, etc?

Thanks

[bharatpotnuri]

I tried to reproduce the problem with a larger number of QPs, and there was no problem. Can you provide the exact command you used? also, it will be helpful to get more information about your setup and chip , driver, etc?

Thanks

Thanks for looking into it. Command is in #54, Here I am pasting it again: [root@saptharishi perftest]# ./ib_write_bw -R -F -n10 -q1026 -s32 --report_gbits 102.1.1.11 RDMA CM event error: Event: RDMA_CM_EVENT_REJECTED; error: -111. This is seen with Chelsio T6 adapter. However I think this is generic issue. As connections or QPs scale, Having just one shared CQ, makes the CQ so huge that adapters run out of resources, Particularly if the resources are limited by HW.