Improve the Linux llvm_build.sh script.

[gw3583]

• Fetch objects instead of pull - pull fails if run in existing directory due to branch checkout.
• Always run the configure / make steps rather than assuming directory presence means they succeeded (they exit quickly if up to date).
• Use -p flag to execute mkdir without error if the directory exists.
• Switch generated build files to Ninja. This introduces an extra build dependency, but makes builds much faster since Ninja uses all available cores by default.

[gw3583]

I'm not sure if any of this is useful. If you're happy with the ninja build dep, it might be worth switching the other CMake builds over to use it too, as they build quite slowly on a multi-core machine at the moment.

[bfiete]

Yay- I was hoping someone would help out with some of this Linux build stuff. I'll check this out and add Ninja to the build prerequisites.

[subramanivmk]

I tried to compile llvm using this patch, since ninja starts number of jobs based on the processor not amount of RAM it will lead to OOM during linking phase as there will be many parallel linking jobs. Devices with less RAM will become unresponsive during linking phase.

[save-buffer]

You can fix OOM error by setting the LLVM_PARALLEL_LINK_JOBS flag when CMake configuring LLVM. I've set it to 8 before on a beefier machine, but maybe a more conservative setting would be 4?

[mundusnine]

Why add a dependency when you can just specify -jN for multithreaded builds with Gnumake ? where N would be the number of cores + 1. We could just receive in the build script the number of cores by the user.

[Green-Sky]

I'd also like the debug build to be optional

~/workspace/Beef/extern$ du -sh *
3,0M    curl
2,2M    hunspell
4,0K    llvm_build.bat
4,0K    llvm_build.sh
32G     llvm_linux_8_0_0
1,6G    llvm_linux_rel_8_0_0
1,8G    llvm-project_8_0_0
64K     toml

[apg360]

Why add a dependency when you can just specify -jN for multithreaded builds with Gnumake ? where N would be the number of cores + 1. We could just receive in the build script the number of cores by the user.

Because Ninja does much more than simply parallelism.

[apg360]

LLVM_PARALLEL_LINK_JOBS

Valid point ..... for illustration, here how it will be : cmake -G Ninja -DLLVM_PARALLEL_LINK_JOBS=1 ../llvm-project_8_0_0/llvm cmake -G Ninja -DLLVM_PARALLEL_LINK_JOBS=1 ../llvm-project_8_0_0/llvm -DCMAKE_BUILD_TYPE:String=Release

I myself doing it like that whenever I compile Beef.

[apg360]

Yay- I was hoping someone would help out with some of this Linux build stuff. I'll check this out and add Ninja to the build prerequisites.

I will, just please confirm me know if you are willing to accept change to use Ninja .... Your approach would be use ninja if available or always use Ninja only.

For info every time I compiled Beef (linux+macOS) I did with Ninja only. I confirm the out of memory issue when linking when not using LLVM_PARALLEL_LINK_JOBS=1

cmake -G Ninja -DLLVM_PARALLEL_LINK_JOBS=1 ../llvm-project_11_0_0/llvm
cmake -G Ninja -DLLVM_PARALLEL_LINK_JOBS=1 ../llvm-project_11_0_0/llvm -DCMAKE_BUILD_TYPE:String=Release

[makinori]

I just modified the script myself to use ninja, I hope this can get fixed and merged