FedML-AI
How to assign multi-cores to one client (or one process of MPI) when trainning only use CPU?
FedML starts with command of ‘mpirun’, and only assigns one cpu core to each process. As far as I know, 'mpi4py' used by FedML can not support multi-cores per process, and thus limits the performance of model training on each client.
Hence, is there any suggestion to solve this problem?
@zuoxiaojiang Hi, the number of CPU cores used by the process launched by MPI is OS-dependent. See the discussion here: https://stackoverflow.com/questions/5797615/mpi-cores-or-processors
Say you have N CPUs and each CPU has M cores, you will have NM cores. If you create K processed using MPI, the number of cores per process will be (NM)/K.
OK～ Thank you.
发自我的iPhone
------------------ Original ------------------ From: Chaoyang He @.> Date: Sat,Apr 16,2022 3:19 AM To: FedML-AI/FedML @.> Cc: zuoxiaojiang @.>, Mention @.> Subject: Re: [FedML-AI/FedML] How to assign multi-cores to one client (or oneprocess of MPI) when trainning only use CPU? (Issue #173)
Say you have N CPUs and each CPU has M cores, you will have NM cores. If you create K processed using MPI, the number of cores per process will be (NM)/K.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>