cling icon indicating copy to clipboard operation
cling copied to clipboard

Add option in cpt.py to control the number of cores used during make

Open sudo-panda opened this issue 3 years ago • 4 comments

Running make with all cores while building might not be feasible for many. Like in my case the linker crashes due to the lack of memory when all my cores are being used by make. One can obviously edit the file to force the number of cores but having an argument to control the same would be better imo.

sudo-panda avatar Mar 03 '21 15:03 sudo-panda

Have you tried -DLLVM_PARALLEL_LINK_JOBS=<some value> via the cmake extra args flag?

reikdas avatar Mar 03 '21 17:03 reikdas

@reikdas thanks for the reply, I didn't know about that flag. I tried using that but the linker still crashes. :(

Also my computer freezes up frequently during the installation and restricting make to use a particular number of cores seems to be the way to fix that.

sudo-panda avatar Mar 04 '21 07:03 sudo-panda

I'm not sure, which options the cpt.py tool uses, because I build cling via cmake. But my experience are:

  • if you build cling with static libraries (default of cmake) it consumes much memory
  • debug builds increases the memory consumption of the linker very much
  • simultaneous multi threading can causes easily a out-of-memory problems (after a long freeze on Linux)

For example, our workstation system has a 14 core Intel Xeon with SMT (28 threads) and 128 GB ram. I use 28 threads for compiling and 14 threads for linking. In the worst case (static libraries and debug build) all linkers consumes together 70GB RAM on the same time.

SimeonEhrig avatar Mar 04 '21 09:03 SimeonEhrig

@SimeonEhrig it's the same reason that I think supporting an option to control the number of cores used to build while using cpt.py is a good idea.

sudo-panda avatar Mar 08 '21 18:03 sudo-panda