mpich icon indicating copy to clipboard operation
mpich copied to clipboard
verified publisher icon pmodels

Is there no IB channel in the mpich3 release?

Open cs0731 opened this issue 3 years ago • 1 comments

I tried to deploy mpich3 in the Infiniband HDR network. Everything was normal in the deployment process, but the point-to-point bandwidth test program(osu benchmark) was only 170MB/s. If the test bandwidth is used through UCX + mpich3, it is expected. Is there no IB channel in the mpich3 release?

cs0731 avatar Sep 02 '22 11:09 cs0731

Which MPICH release are you referring? If you are using mpich-3.2, you may try --with-device=ch3:nemesis:ofi. ch4:ucx should be available in mpich-3.3 and newer.

hzhou avatar Sep 03 '22 04:09 hzhou

Which MPICH release are you referring? If you are using mpich-3.2, you may try --with-device=ch3:nemesis:ofi. ch4:ucx should be available in mpich-3.3 and newer.

I deployed mpich-3.3.2 and libfabric version 1.15.2. Then I tried to use "-- with device=ch3: nemesis: ofi". The measured bandwidth of osu_bw is only 5GB/s, far below the expected performance.

cs0731 avatar Sep 26 '22 15:09 cs0731

@cs0731 ch3 is in maintenance mode. Is it possible for you try ch4 using later MPICH releases?

hzhou avatar Sep 26 '22 15:09 hzhou

@cs0731 ch3 is in maintenance mode. Is it possible for you try ch4 using later MPICH releases?

Thank you! I have deployed mpich-3.3.2. Using ch4: ucx, the network performance test is normal.

cs0731 avatar Sep 27 '22 14:09 cs0731