llama.cpp
llama.cpp copied to clipboard
ggml-org
Add detection code for avx/avx2/etc
Add the code to check the build host to determine the right CPU feature.
This is convenient when build Windows version on the machine without AVX2.
This seems to work fine here on Linux, but it's not really needed here as we have -march=native. I don't have Windows to test, so someone else should probably have a look.
CHECK_SSE should better be called CHECK_SIMD, as SSE is a specific instruction set, the predecessor to AVX.
A disadvantage to this PR is if the build output is not intended to be run on the build host, e.g. your build host doesn't have AVX512 but you want to build with it enabled in order to run it on another machine. This would prevent that.
As far as I understand it, the failed checks I'm seeing are due to it trying the MSVC flags on GCC. I guess you could separate the tests by compiler, but that would make it more complicated.
Maybe a better solution would be to have a LLAMA_PORTABLE option in cmake.
If you set PORTABLE=OFF:
- on gcc it will set
-march=native - on MSVC it will run your checks, but just for the MSVC flags, and enable all that succeed
If you set PORTABLE=ON, you'll have to specify the options you want (LLAMA_AVX2 etc.)
(Instead of LLAMA_PORTABLE, we might also have something like LLAMA_NATIVE with opposite meaning)
@sw , I can confirm this works on Windows 10 I debugged a crash in quantize that was due to AVX not supported on Intel Pentium G3220 CPU. I thought of fixing the AVX flag in CMakeLists.txt, and decided to look for existing issues / PRs first. I've found this one and can confirm it works well to detect the missing AVX flag, and that quantize is now working properly
I decided to do the following:
- only do the check while LLAMA_NATIVE and MSVC both are TRUE.
- Leave LLAMA_NATIVE as default OFF to keep the current behavior. However I think it is better to ON as most users are building for its own box.
- Didn't introduce _PORTABLE as NATIVE is an existing variable. keep it simple.
only do the check while LLAMA_NATIVE and MSVC both are TRUE.
Does this mean that this fix is only for Windows? Can you make it also work on Linux?
Yes please. With the wrong CPU I just get: signal: illegal instruction (core dumped).. Crash!
