compas
How to improve the usage of computer on rmcdhf_mpi calculation?
Hi everyone.
Last time I asked a question about rangular calculation, @jongrumer suggested me to expand the memory and I have expanded it to 256GB. But when I use the rmcdhf_mpi program, I find the usage of the computer is very low. Both the CPU and memory are not fully used. The top command shows below:
top - 21:52:55 up 15:20, 2 users, load average: 12.14, 10.82, 6.61
Tasks: 321 total, 2 running, 319 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.7 us, 0.5 sy, 0.0 ni, 49.2 id, 46.6 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 26377857+total, 6408952 free, 20250176 used, 23711945+buff/cache
KiB Swap: 16383996 total, 16273660 free, 110336 used. 24256968+avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
49406 grasp20+ 20 0 1164420 725968 5716 D 5.6 0.3 9:43.40 rmcdhf_mpi
49402 grasp20+ 20 0 1164632 727820 7320 D 5.3 0.3 9:42.68 rmcdhf_mpi
49426 grasp20+ 20 0 1164012 725900 6008 D 5.3 0.3 9:44.11 rmcdhf_mpi
49404 grasp20+ 20 0 1164040 725600 5712 R 5.0 0.3 9:43.59 rmcdhf_mpi
49411 grasp20+ 20 0 1163208 724724 5672 D 4.6 0.3 9:43.65 rmcdhf_mpi
49418 grasp20+ 20 0 1163524 725100 5724 D 4.6 0.3 9:40.46 rmcdhf_mpi
49421 grasp20+ 20 0 1164064 725668 5724 D 3.6 0.3 9:42.43 rmcdhf_mpi
49401 grasp20+ 20 0 1163424 729668 10616 D 3.3 0.3 1:22.29 rmcdhf_mpi
49408 grasp20+ 20 0 1163784 725328 5712 D 3.3 0.3 9:42.92 rmcdhf_mpi
49403 grasp20+ 20 0 1163336 726488 7296 D 3.0 0.3 9:42.50 rmcdhf_mpi
49415 grasp20+ 20 0 1163572 725832 6412 D 3.0 0.3 9:40.70 rmcdhf_mpi
49405 grasp20+ 20 0 1163268 725652 6548 D 2.6 0.3 9:43.03 rmcdhf_mpi
11545 grasp20+ 20 0 162244 2500 1564 R 1.0 0.0 3:32.81 top
Is there anything I can do to improve the usage of the computer?
Have a nice day!
Yenoch
Hi @YenochQin, if the CPU usage goes down you should check the disk reading/writing - the RMCDHF code is reading the spin-angular coefficients from the mcp-files calculated by the RANGULAR code, which can be very large for complex cases with a lot of interactions (dense matrices). You can monitor the disk i/o with iotop (might require sudo). If you have access to a fast SSD disk instead of a regular mechanical HDD, that typically speeds up the calculation significantly.
You can of course also try to make the RMCDHF calculation smaller, more compact, and then expand the CSF-space in the subsequent RCI calculation - which is much faster (essentially one loop of RMCDHF).
The program could be modified to store the data in memory, if possible, and only read from disk when not enough memory is available.
Thank you for the replies. It's true that the rmcdhf_mpi program need fast SSD disks. I tested on another HCP, which have SSD storage. All the CPU cores are fully used. So I will go to add some SSDs on our team's HPC.
@cffischer Could you please tell me how to modify the program to store the data in memory? Editing the code or changing the parameters when running the program?
@YenochQin Fortran supports both external and internal files. The simplest change would be for each process to copyi its external files to internal files so that during the scf iteration , the reading is from internal files in memory. It has been a while since I did this but I believe this means that each process needs to allocate memory for the file data. So , ideally, rangular needs to output a files about the size of data arrays. So maybe rangular needs to output a file,, say mcp.inf (for information), the rmcdhf then used to allocate arrays. I have an rcsfgen that produces an rcsf.inf file for information that rangular can use.
Let me know if you need more help.
@cffischer Thank you for your reply again. : ) After reading your introduction, I think it could be a big challenge for me due to the lack of ability about Fortran coding. So I would like to add SSD disks to our HPC.