ggml-org
Compile bug: How to compile llama.cpp with Vulkan for android device
Git commit
902368a
Operating systems
Linux
GGML backends
Vulkan
Problem description & steps to reproduce
I tried to compile llama.cpp(b4644) using NDK 27 and Vulkan-header(v1.4.307) and encountered the following compilation issues.
First Bad Commit
No response
Compile command
cmake \
-DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
-DANDROID_ABI=arm64-v8a \
-DANDROID_PLATFORM=latest \
-DCMAKE_C_FLAGS=-march=armv8.4a+dotprod \
-DGGML_VULKAN=ON \
-DGGML_VULKAN_CHECK_RESULTS=OFF \
-DGGML_VULKAN_DEBUG=ON \
-DGGML_VULKAN_MEMORY_DEBUG=ON \
-DGGML_VULKAN_SHADER_DEBUG_INFO=ON \
-DGGML_VULKAN_PERF=OFF \
-DGGML_VULKAN_VALIDATE=OFF \
-DGGML_VULKAN_RUN_TESTS=OFF \
-DVK_USE_PLATFORM_ANDROID_KHR=ON \
-DVulkan_INCLUDE_DIR=$Vulkan-Headers/include
-DGGML_CCACHE=OFF \
-B build-android
Relevant log output
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1378:2: warning: extra ';' outside of a function is incompatible with C++98 [-Wc++98-compat-extra-semi]
1378 | };
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
1607 | CREATE_FA(GGML_TYPE_F16, f16)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:46:1: note: expanded from here
46 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:48:1: note: expanded from here
48 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]); \
| ^
<scratch space>:51:1: note: expanded from here
51 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]); \
| ^
<scratch space>:53:1: note: expanded from here
53 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:56:1: note: expanded from here
56 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:214: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:58:1: note: expanded from here
58 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:163: note: expanded from macro 'CREATE_FA2'
1593 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]); \
| ^
<scratch space>:61:1: note: expanded from here
61 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:214: note: expanded from macro 'CREATE_FA2'
1593 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]); \
| ^
<scratch space>:63:1: note: expanded from here
63 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:163: note: expanded from macro 'CREATE_FA2'
1594 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1); \
| ^
<scratch space>:66:1: note: expanded from here
66 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:214: note: expanded from macro 'CREATE_FA2'
1594 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1); \
| ^
<scratch space>:68:1: note: expanded from here
68 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:163: note: expanded from macro 'CREATE_FA2'
1595 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]); \
| ^
<scratch space>:71:1: note: expanded from here
71 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:214: note: expanded from macro 'CREATE_FA2'
1595 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]); \
| ^
<scratch space>:73:1: note: expanded from here
73 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:163: note: expanded from macro 'CREATE_FA2'
1596 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1); \
| ^
<scratch space>:76:1: note: expanded from here
76 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:214: note: expanded from macro 'CREATE_FA2'
1596 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1); \
| ^
<scratch space>:78:1: note: expanded from here
78 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:163: note: expanded from macro 'CREATE_FA2'
1597 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]); \
| ^
<scratch space>:81:1: note: expanded from here
81 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:214: note: expanded from macro 'CREATE_FA2'
1597 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]); \
| ^
<scratch space>:83:1: note: expanded from here
83 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 80) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:102:1: note: expanded from here
102 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 80) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:104:1: note: expanded from here
104 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 80) \
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]); \
| ^
<scratch space>:107:1: note: expanded from here
107 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
gmake[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/build.make:160: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [CMakeFiles/Makefile2:1795: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
Please attach the full log for a clean build. This looks like another "weird combination of tooling installed" issue.
@jeffbolznv please take a look.
echo $ANDROID_NDK
/build/Android/Sdk/ndk/28.0.12674087/
b4651
git status
HEAD detached at b4651
nothing to commit, working tree clean
cmake -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-33 -DCMAKE_C_FLAGS="-march=armv8.7a" -DCMAKE_CXX_FLAGS="-march=armv8.7a" -DGGML_VULKAN=ON -DVulkan_INCLUDE_DIR=$VULKAN/include -B build-android
-- The C compiler identification is Clang 19.0.0
-- The CXX compiler identification is Clang 19.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /build/Android/Sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /build/Android/Sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.48.1")
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp=libomp
-- Found OpenMP_CXX: -fopenmp=libomp
-- Found OpenMP: TRUE
-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- Adding CPU backend variant ggml-cpu:
-- Found Vulkan: /build/Android/Sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/33/libvulkan.so (found version "1.3.302") found components: glslc missing components: glslangValidator
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 supported by glslc
-- Host compiler: /usr/bin/gcc /usr/bin/g++
-- vulkan-shaders-gen toolchain file: /build/prj/test/llama.cpp/build-android/host-toolchain.cmake
-- Including Vulkan backend
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Configuring done (1.3s)
-- Generating done (0.1s)
-- Build files have been written to: /build/prj/test/llama.cpp/build-android
cmake --build build-android/ --config Release
[ 1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[ 1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[ 2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[ 2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[ 3%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[ 3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[ 4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[ 4%] Linking CXX shared library ../../bin/libggml-base.so
[ 4%] Built target ggml-base
[ 5%] Creating directories for 'vulkan-shaders-gen'
[ 5%] No download step for 'vulkan-shaders-gen'
[ 6%] No update step for 'vulkan-shaders-gen'
[ 6%] No patch step for 'vulkan-shaders-gen'
[ 7%] Performing configure step for 'vulkan-shaders-gen'
CMake Warning (dev) in CMakeLists.txt:
No project() command is present. The top-level CMakeLists.txt file must
contain a literal, direct call to the project() command. Add a line of
code such as
project(ProjectName)
near the top of the file, but after cmake_minimum_required().
CMake is pretending there is a "project(Project)" command on the first
line.
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) in CMakeLists.txt:
cmake_minimum_required() should be called prior to this top-level project()
call. Please see the cmake-commands(7) manual for usage documentation of
both commands.
This warning is for project developers. Use -Wno-dev to suppress it.
-- The C compiler identification is GNU 14.2.1
-- The CXX compiler identification is GNU 14.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as
cmake_minimum_required(VERSION 3.27)
should be added at the top of the file. The version specified may be lower
if you wish to support older CMake versions for this project. For more
information run "cmake --help-policy CMP0000".
This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done (0.4s)
-- Generating done (0.0s)
-- Build files have been written to: /build/prj/test/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[ 7%] Performing build step for 'vulkan-shaders-gen'
[ 50%] Building CXX object CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.o
[100%] Linking CXX executable /build/prj/test/llama.cpp/build-android/bin/vulkan-shaders-gen
[100%] Built target vulkan-shaders-gen
[ 7%] Built target vulkan-shaders-gen-build
[ 7%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "Release"
[ 7%] Built target vulkan-shaders-gen-install
[ 8%] Completed 'vulkan-shaders-gen'
[ 8%] Built target vulkan-shaders-gen
[ 8%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
[ 8%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1379:2: warning: extra ';' outside of a function is incompatible with C++98 [-Wc++98-compat-extra-semi]
1379 | };
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
1608 | CREATE_FA(GGML_TYPE_F16, f16)
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:212:1: note: expanded from here
212 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:214:1: note: expanded from here
214 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]); \
| ^
<scratch space>:217:1: note: expanded from here
217 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:214: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]); \
| ^
<scratch space>:219:1: note: expanded from here
219 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:163: note: expanded from macro 'CREATE_FA2'
1593 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:222:1: note: expanded from here
222 | flash_attn_f32_f16_f16_cm2_len
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:214: note: expanded from macro 'CREATE_FA2'
1593 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:224:1: note: expanded from here
224 | flash_attn_f32_f16_f16_cm2_data
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:163: note: expanded from macro 'CREATE_FA2'
1594 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]); \
| ^
<scratch space>:227:1: note: expanded from here
227 | flash_attn_f32_f16_f16_cm2_len
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:214: note: expanded from macro 'CREATE_FA2'
1594 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]); \
| ^
<scratch space>:229:1: note: expanded from here
229 | flash_attn_f32_f16_f16_cm2_data
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:163: note: expanded from macro 'CREATE_FA2'
1595 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1); \
| ^
<scratch space>:232:1: note: expanded from here
232 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:214: note: expanded from macro 'CREATE_FA2'
1595 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1); \
| ^
<scratch space>:234:1: note: expanded from here
234 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:163: note: expanded from macro 'CREATE_FA2'
1596 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]); \
| ^
<scratch space>:237:1: note: expanded from here
237 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:214: note: expanded from macro 'CREATE_FA2'
1596 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]); \
| ^
<scratch space>:239:1: note: expanded from here
239 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:163: note: expanded from macro 'CREATE_FA2'
1597 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1); \
| ^
<scratch space>:242:1: note: expanded from here
242 | flash_attn_f32_f16_f16_cm2_len
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:214: note: expanded from macro 'CREATE_FA2'
1597 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1); \
| ^
<scratch space>:244:1: note: expanded from here
244 | flash_attn_f32_f16_f16_cm2_data
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1598:163: note: expanded from macro 'CREATE_FA2'
1598 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]); \
| ^
<scratch space>:247:1: note: expanded from here
247 | flash_attn_f32_f16_f16_cm2_len
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 64) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1598:214: note: expanded from macro 'CREATE_FA2'
1598 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len, flash_attn_f32_f16_ ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]); \
| ^
<scratch space>:2:1: note: expanded from here
2 | flash_attn_f32_f16_f16_cm2_data
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1602:9: note: expanded from macro 'CREATE_FA'
1602 | CREATE_FA2(TYPE, NAMELC, 80) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:21:1: note: expanded from here
21 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1602:9: note: expanded from macro 'CREATE_FA'
1602 | CREATE_FA2(TYPE, NAMELC, 80) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1); \
| ^
<scratch space>:23:1: note: expanded from here
23 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1602:9: note: expanded from macro 'CREATE_FA'
1602 | CREATE_FA2(TYPE, NAMELC, 80) \
| ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]); \
| ^
<scratch space>:26:1: note: expanded from here
26 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
gmake[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/build.make:161: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1795: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
Please attach the full log for a clean build. This looks like another "weird combination of tooling installed" issue.
Hi, here are all the logs from when I try to compile the Vulkan and llama.cpp for Android.
~/test/llama.cpp-b4644$ cmake
-DCMAKE_TOOLCHAIN_FILE=/home/ubuntu/android-ndk-r27c/build/cmake/android.toolchain.cmake
-DANDROID_ABI=arm64-v8a
-DANDROID_PLATFORM=android-28
-DCMAKE_C_FLAGS="-march=armv8.7a"
-DCMAKE_CXX_FLAGS="-march=armv8.7a"
-DGGML_OPENMP=OFF
-DGGML_LLAMAFILE=OFF
-DGGML_VULKAN=1
-DVulkan_INCLUDE_DIR=/home/ubuntu/Vulkan-Headers/include
-DVulkan_LIBRARY=/home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/35/libvulkan.so
-B build-android
-- The C compiler identification is Clang 18.0.3
-- The CXX compiler identification is Clang 18.0.3
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.34.1")
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- Adding CPU backend variant ggml-cpu:
-- Found Vulkan: /home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/35/libvulkan.so
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 supported by glslc
-- Host compiler: /usr/bin/gcc /usr/bin/g++
-- vulkan-shaders-gen toolchain file: /home/ubuntu/test/llama.cpp-b4644/build-android/host-toolchain.cmake
-- Including Vulkan backend
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
CMake Warning at common/CMakeLists.txt:32 (message):
Git repository not found; to enable automatic generation of build info,
make sure Git is installed and the project is a Git repository.
-- Configuring done -- Generating done -- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android
~/test/llama.cpp-b4644$ cmake --build build-android --config Release -j8 [ 0%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o [ 0%] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o [ 1%] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o [ 2%] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o [ 3%] Creating directories for 'vulkan-shaders-gen' [ 4%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o [ 5%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o [ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o [ 5%] Built target build_info [ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o [ 5%] No download step for 'vulkan-shaders-gen' [ 6%] No update step for 'vulkan-shaders-gen' [ 6%] Built target sha1 [ 7%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o [ 7%] Built target sha256 [ 7%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o [ 7%] No patch step for 'vulkan-shaders-gen' [ 8%] Performing configure step for 'vulkan-shaders-gen' CMake Warning (dev) in CMakeLists.txt: No project() command is present. The top-level CMakeLists.txt file must contain a literal, direct call to the project() command. Add a line of code such as
project(ProjectName)
near the top of the file, but after cmake_minimum_required().
CMake is pretending there is a "project(Project)" command on the first line. This warning is for project developers. Use -Wno-dev to suppress it.
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
[ 9%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as
cmake_minimum_required(VERSION 3.22)
should be added at the top of the file. The version specified may be lower if you wish to support older CMake versions for this project. For more information run "cmake --help-policy CMP0000". This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[ 9%] Performing build step for 'vulkan-shaders-gen'
gmake[3]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
[ 50%] Building CXX object CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.o
[ 9%] Built target xxhash
[100%] Linking CXX executable /home/ubuntu/test/llama.cpp-b4644/build-android/bin/vulkan-shaders-gen
[100%] Built target vulkan-shaders-gen
[ 9%] Built target vulkan-shaders-gen-build
[ 9%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "Release"
[ 9%] Built target vulkan-shaders-gen-install
[ 10%] Completed 'vulkan-shaders-gen'
[ 10%] Built target vulkan-shaders-gen
[ 10%] Linking CXX shared library ../../bin/libggml-base.so
[ 10%] Built target ggml-base
[ 11%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
[ 13%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[ 13%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[ 14%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o
[ 15%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
[ 15%] Linking CXX shared library ../../bin/libggml-cpu.so
[ 15%] Built target ggml-cpu
[ 16%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o
[ 16%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan-shaders.cpp.o
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1378:2: warning: extra ';' outside of a function is incompatible with C++98 [-Wc++98-compat-extra-semi]
1378 | };
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
1607 | CREATE_FA(GGML_TYPE_F16, f16)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "aligned_f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "aligned_f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:214: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:163: note: expanded from macro 'CREATE_FA2'
1593 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "aligned_f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:214: note: expanded from macro 'CREATE_FA2'
1593 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "aligned_f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:163: note: expanded from macro 'CREATE_FA2'
1594 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:214: note: expanded from macro 'CREATE_FA2'
1594 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:163: note: expanded from macro 'CREATE_FA2'
1595 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:214: note: expanded from macro 'CREATE_FA2'
1595 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:163: note: expanded from macro 'CREATE_FA2'
1596 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:214: note: expanded from macro 'CREATE_FA2'
1596 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:163: note: expanded from macro 'CREATE_FA2'
1597 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:214: note: expanded from macro 'CREATE_FA2'
1597 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "aligned_f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
OK, so cmake detects coopmat2 support, but those shader are either not being generated at all, or the generation is silently crashing (seems less likely). Please check whether GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is defined when vulkan-shaders-gen.cpp is compiled. I'm thinking this could be an issue with cross compiles not picking up the cmake state.
OK, so cmake detects coopmat2 support, but those shader are either not being generated at all, or the generation is silently crashing (seems less likely). Please check whether
GGML_VULKAN_COOPMAT2_GLSLC_SUPPORTis defined when vulkan-shaders-gen.cpp is compiled. I'm thinking this could be an issue with cross compiles not picking up the cmake state.
Hi, GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT was not defined when compiling vulkan-shaders-gen.cpp. However, when I add #ifdef GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT #pragma message("GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is defined") #else #pragma message("GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is NOT defined") #define GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT #endif at the beginning of vulkan-shaders-gen.cpp, it still has the same compilation error. Can you help check again?
at the beginning of vulkan-shaders-gen.cpp, it still has the same compilation error. Can you help check again?
That's surprising. Was it a clean build? I don't fully trust the dependencies (they're manually hacked together in _ggml_vk_shader_deps).
GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT gets set in ggml-vulkan/CMakeLists.txt and in normal builds that setting gets inherited into the vulkan-shaders subproject. It sounds like this cross-compile may not be getting that behavior.
Either that (or in addition to that!) the cross-compile might be picking up a different glslc that does not support coopmat2 (seems very possible if the ndk hasn't updated). Is it possible to check what version of glslc is in the NDK and also try using it to compile -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp
I updated the nkd to the latest version
/build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp
vulkan-shaders/test_coopmat_support.comp:3: error: '#extension' : extension not supported: GL_KHR_cooperative_matrix
1 error generated.
hardcoding glslc executable to the one provided by ndk solves the issue.
diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
index 07403108..4e7271ff 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
@@ -1,5 +1,5 @@
find_package (Threads REQUIRED)
-find_program(GLSLC_EXECUTABLE glslc)
+find_program(GLSLC_EXECUTABLE /build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc)
if(NOT GLSLC_EXECUTABLE)
message(FATAL_ERROR "glslc not found.")
endif()
I updated the nkd to the latest version
/build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp vulkan-shaders/test_coopmat_support.comp:3: error: '#extension' : extension not supported: GL_KHR_cooperative_matrix 1 error generated.hardcoding
glslcexecutable to the one provided by ndk solves the issue.diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt index 07403108..4e7271ff 100644 --- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt +++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt @@ -1,5 +1,5 @@ find_package (Threads REQUIRED) -find_program(GLSLC_EXECUTABLE glslc) +find_program(GLSLC_EXECUTABLE /build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc) if(NOT GLSLC_EXECUTABLE) message(FATAL_ERROR "glslc not found.") endif()
Hi, which version of llama.cpp did you compile, when I use this way and the latest NDK, the problem still exists.
at the beginning of vulkan-shaders-gen.cpp, it still has the same compilation error. Can you help check again?
That's surprising. Was it a clean build? I don't fully trust the dependencies (they're manually hacked together in
_ggml_vk_shader_deps).
GGML_VULKAN_COOPMAT2_GLSLC_SUPPORTgets set inggml-vulkan/CMakeLists.txtand in normal builds that setting gets inherited into thevulkan-shaderssubproject. It sounds like this cross-compile may not be getting that behavior.Either that (or in addition to that!) the cross-compile might be picking up a different glslc that does not support coopmat2 (seems very possible if the ndk hasn't updated). Is it possible to check what version of glslc is in the NDK and also try using it to compile
-o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp
Hi, I use clean build every time. When downgrading vulkan from 1.4 to 1.3. The compilation log is as follows:
cmake --build build-android/ --target clean
~/test/llama.cpp-b4644$ cmake -DCMAKE_TOOLCHAIN_FILE=/home/ubuntu/android-ndk-r27c/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-28 -DCMAKE_C_FLAGS="-march=armv8.7a" -DCMAKE_CXX_FLAGS="-march=armv8.7a" -DGGML_OPENMP=OFF -DGGML_LLAMAFILE=OFF -DGGML_VULKAN=1 -DVulkan_INCLUDE_DIR=/home/ubuntu/test/Vulkan-Headers/include -DVulkan_LIBRARY=/home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/35/libvulkan.so -B build-android
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- ARM detected
-- Adding CPU backend variant ggml-cpu:
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 not supported by glslc
-- Host compiler: /usr/bin/gcc /usr/bin/g++
-- vulkan-shaders-gen toolchain file: /home/ubuntu/test/llama.cpp-b4644/build-android/host-toolchain.cmake
-- Including Vulkan backend
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
CMake Warning at common/CMakeLists.txt:32 (message):
Git repository not found; to enable automatic generation of build info,
make sure Git is installed and the project is a Git repository.
-- Configuring done -- Generating done -- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android ~/test/llama.cpp-b4644$ cmake --build build-android --config Release -j8 [ 0%] Generating build details from Git [ 1%] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o [ 2%] Creating directories for 'vulkan-shaders-gen' [ 3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o -- Found Git: /usr/bin/git (found version "2.34.1") [ 4%] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o [ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o [ 5%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o fatal: not a git repository (or any of the parent directories): .git [ 5%] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o fatal: not a git repository (or any of the parent directories): .git [ 5%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o [ 5%] No download step for 'vulkan-shaders-gen' [ 5%] Built target build_info [ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o [ 6%] No update step for 'vulkan-shaders-gen' [ 6%] No patch step for 'vulkan-shaders-gen' [ 6%] Built target sha1 [ 7%] Performing configure step for 'vulkan-shaders-gen' [ 8%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o CMake Warning (dev) in CMakeLists.txt: No project() command is present. The top-level CMakeLists.txt file must contain a literal, direct call to the project() command. Add a line of code such as
project(ProjectName)
near the top of the file, but after cmake_minimum_required().
CMake is pretending there is a "project(Project)" command on the first line. This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) in CMakeLists.txt: No cmake_minimum_required command is present. A line of code such as
cmake_minimum_required(VERSION 3.22)
should be added at the top of the file. The version specified may be lower if you wish to support older CMake versions for this project. For more information run "cmake --help-policy CMP0000". This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done -- Generating done -- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build [ 8%] Built target sha256 [ 8%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o [ 8%] Performing build step for 'vulkan-shaders-gen' gmake[3]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule. Consolidate compiler generated dependencies of target vulkan-shaders-gen [100%] Built target vulkan-shaders-gen [ 8%] Built target vulkan-shaders-gen-build [ 9%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o [ 9%] Performing install step for 'vulkan-shaders-gen' -- Install configuration: "Release" [ 9%] Built target vulkan-shaders-gen-install [ 10%] Completed 'vulkan-shaders-gen' [ 10%] Built target vulkan-shaders-gen [ 10%] Built target xxhash [ 10%] Linking CXX shared library ../../bin/libggml-base.so [ 10%] Built target ggml-base [ 11%] Generate vulkan shaders ggml_vulkan: Generating and compiling shaders to SPIR-V [ 11%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o [ 12%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o [ 12%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o [ 12%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o [ 13%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o [ 13%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o [ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o [ 15%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o cannot compile matmul_f32_f16_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_cm2.spv -DACC_TYPE=float -DB_TYPE=float16_t -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_f32_f16_aligned_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=float16_t -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_f16_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_cm2.spv -DACC_TYPE=float -DB_TYPE=float16_t -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q4_1_f16_aligned_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q4_0_f16_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q4_1_f16_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_f16_aligned_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=float16_t -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q5_0_f16_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_0_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q5_1_f16_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_1_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q4_0_f16_aligned_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q5_0_f16_aligned_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_0_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q5_1_f16_aligned_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_1_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q8_0_f16_aligned_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q8_0_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q8_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q8_0_f16_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q8_0_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q8_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q2_k_f16_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q2_k_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q2_K=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
cannot compile matmul_q2_k_f16_aligned_cm2
/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q2_k_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q2_K=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2 /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion) /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON 1 warning and 4 errors generated.
[ 15%] Linking CXX shared library ../../bin/libggml-cpu.so [ 15%] Built target ggml-cpu
And the compilation process will be blocked at [ 15%] Built target ggml-cpu.
/build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp vulkan-shaders/test_coopmat_support.comp:3: error: '#extension' : extension not supported: GL_KHR_cooperative_matrix 1 error generated.
OK, sounds like the ndk glslc is very old (GL_KHR_cooperative_matrix was release in June 2023). And the issue seems to be related to vulkan-shaders/CMakeLists.txt picking up a different glslc than ggml-vulkan/CMakeLists.txt.
hardcoding
glslcexecutable to the one provided by ndk solves the issue.diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt index 07403108..4e7271ff 100644 --- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt +++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt @@ -1,5 +1,5 @@ find_package (Threads REQUIRED) -find_program(GLSLC_EXECUTABLE glslc) +find_program(GLSLC_EXECUTABLE /build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc) if(NOT GLSLC_EXECUTABLE) message(FATAL_ERROR "glslc not found.") endif()
This is surprising. If this glslc doesn't support KHR_coopmat, I'd expect those compiles to fail. Does this somehow affect the logic in ggml-vulkan/CMakeLists.txt? I'm not an expert on cmake.
But it seems like there are two glslcs in the system, the one from the ndk is old, the two CMakeLists.txt files are picking up different versions, and I suspect this is somehow related to the cross-compiling or overriding the toolchain in ggml-vulkan/CMakeLists.txt.
Hi, I use clean build every time. When downgrading vulkan from 1.4 to 1.3. The compilation log is as follows: ... -- GL_NV_cooperative_matrix2 not supported by glslc
This downgrade picked up a different glslc that doesn't support coopmat2 and this avoids the problem. We need the two CMakeLists.txt files to agree on which build of glslc is being used.
(Or, we need to stop relying on whatever random stuff is installed and fetch and build shaderc directly. This just keeps coming up).
I spent some time debugging the issue
CMake Debug Log at ggml/src/ggml-vulkan/CMakeLists.txt:5 (find_package):
find_package considered the following paths for FindVulkan.cmake:
/buid/prj/test/llama.cpp/cmake/FindVulkan.cmake
The file was found at
/build/.local/lib/python3.10/site-packages/cmake/data/share/cmake-3.27/Modules/FindVulkan.cmake
added a debug message to check what gets resolved
iff --git a/ggml/src/ggml-vulkan/CMakeLists.txt b/ggml/src/ggml-vulkan/CMakeLists.txt
index d970f7e2..43c3c244 100644
--- a/ggml/src/ggml-vulkan/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/CMakeLists.txt
@@ -17,6 +26,7 @@ endfunction()
if (Vulkan_FOUND)
message(STATUS "Vulkan found")
+ message(STATUS Vulkan::Vulkan: ${Vulkan_INCLUDE_DIRS} : ${Vulkan_LIBRARIES} : ${Vulkan_GLSLC_EXECUTABLE})
here is the result:
-- Vulkan::Vulkan:/build/prj/vulkan/Vulkan-Headers-1.3.302/include:/build/Android/Sdk/ndk/28.0.13004108/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/33/libvulkan.so:/bin/glslc
so cmake properly detects path to headers and the library, but fails to lookup the right glslc executable, it simply doesn't look inside $ANDROID_NDK/shader-tools/linux-x86_64/
as a potential workaround, we can have a patched copy of FindVulkan.cmake in the repo under cmake folder which is capable to find the right glslc in case we cross-compile with android ndk toolchain.
@jeffbolznv I can put together a quick PR, please lmk if the proposed approach makes sense
Having a copy of a core cmake module seems hacky and potentially has licensing issues.
I can think of a couple options for dealing with this: (1) Have ggml-vulkan lookup glslc and put the executable name in a variable that vulkan-shaders uses. This way they use the same version and agree on what extensions are supported. Doesn't matter if this is the installed version or the NDK version (so I assume it would get the installed version). (2) Let them continue to use different versions of GLSLC, but have the makefiles communicate to each other which extensions are supported, and only enable those supported by both.
Option (1) seems much easier, so that's what I think we should try.
When I hardcode /home/ubuntu/android-ndk-r28/shader-tools/linux-x86_64/glslc in ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt and add -DVulkan_GLSLC_EXECUTABLE=/home/ubuntu/android-ndk-r28/shader-tools/linux-x86_64/glslc in cmake command, compilation succeeds.
New cmake command is
cmake
-DCMAKE_TOOLCHAIN_FILE=/home/ubuntu/android-ndk-r28/build/cmake/android.toolchain.cmake
-DANDROID_ABI=arm64-v8a
-DANDROID_PLATFORM=android-33
-DCMAKE_C_FLAGS="-march=armv8.7a"
-DCMAKE_CXX_FLAGS="-march=armv8.7a"
-DGGML_VULKAN=ON
-DVulkan_INCLUDE_DIR=/home/ubuntu/Vulkan-Headers/include
-DVulkan_GLSLC_EXECUTABLE=/home/ubuntu/android-ndk-r28/shader-tools/linux-x86_64/glslc
-B build-android
When I try to run LD_LIBRARY_PATH=/data/local/tmp/mobile-vulkan/lib ./bin/llama-bench -m ../model/Llama-3.2-1B-Instruct-Q4_K_M.gguf on Android device, the result shows "CANNOT LINK EXECUTABLE "./bin/llama-bench": library "libomp.so" not found: needed by /data/local/tmp/mobile-vulkan/lib/libggml-cpu.so in namespace (default)".
After moving libomp.so from ${NDK}/toolchains/llvm/prebuilt/linux-x86_64/lib/clang/19/lib/linux/aarch64/libomp.so to {install-dir}/lib, running "LD_LIBRARY_PATH=/data/local/tmp/mobile-vulkan/lib ./bin/llama-bench -m ../model/Llama-3.2-1B-Instruct-Q8_0.gguf", got: ggml_vulkan: Found 1 Vulkan devices: ggml_vulkan: 0 = Adreno (TM) 750 (Qualcomm Technologies Inc. Adreno Vulkan Driver) | uma: 1 | fp16: 1 | warp size: 64 | shared memory: 32768 | matrix cores: none libc++abi: terminating due to uncaught exception of type vk::DeviceLostError: vk::Queue::submit: ErrorDeviceLost Aborted.
Run "LD_LIBRARY_PATH=/data/local/tmp/mobile-vulkan/lib ./bin/llama-bench -m ../model/Llama-3.2-1B-Instruct-Q4_K_M.gguf", got: ggml_vulkan: Found 1 Vulkan devices: ggml_vulkan: 0 = Adreno (TM) 750 (Qualcomm Technologies Inc. Adreno Vulkan Driver) | uma: 1 | fp16: 1 | warp size: 64 | shared memory: 32768 | matrix cores: none ggml_vulkan: Compute pipeline creation failed for mul_mat_vec_q4_k_f32_f32_1 ggml_vulkan: vk::Device::createComputePipeline: ErrorUnknown libc++abi: terminating due to uncaught exception of type std::out_of_range: unordered_map::at: key not found Aborted.
Is there any way to solve this problem, or is there another compilation method required to solve this runtime error?
Having a copy of a core cmake module seems hacky and potentially has licensing issues.
I can think of a couple options for dealing with this: (1) Have ggml-vulkan lookup glslc and put the executable name in a variable that vulkan-shaders uses. This way they use the same version and agree on what extensions are supported. Doesn't matter if this is the installed version or the NDK version (so I assume it would get the installed version). (2) Let them continue to use different versions of GLSLC, but have the makefiles communicate to each other which extensions are supported, and only enable those supported by both.
Option (1) seems much easier, so that's what I think we should try.
Please check if this branch fixes the glslc mismatch: https://github.com/jeffbolznv/llama.cpp/tree/glslc_mismatch.
-DVulkan_GLSLC_EXECUTABLE=/home/ubuntu/android-ndk-r28/shader-tools/linux-x86_64/glslc
Given how old it is, I kind of think it's a bad idea to force it to use the glslc from the ndk.
Please check if this branch fixes the glslc mismatch: https://github.com/jeffbolznv/llama.cpp/tree/glslc_mismatch.
This is so weird. If I remove -DVulkan_GLSLC_EXECUTABLE=, the compilation information is as follows:
/llama.cpp$ cmake --build build-android --config Release -j8 [ 1%] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o [ 2%] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o [ 2%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o [ 3%] Creating directories for 'vulkan-shaders-gen' [ 3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o [ 4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o [ 4%] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o [ 5%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o [ 5%] Built target build_info [ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o [ 5%] No download step for 'vulkan-shaders-gen' [ 6%] No update step for 'vulkan-shaders-gen' [ 6%] No patch step for 'vulkan-shaders-gen' [ 6%] Built target sha1 [ 7%] Performing configure step for 'vulkan-shaders-gen' [ 8%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o CMake Warning (dev) in CMakeLists.txt: No project() command is present. The top-level CMakeLists.txt file must contain a literal, direct call to the project() command. Add a line of code such as
project(ProjectName)
near the top of the file, but after cmake_minimum_required().
CMake is pretending there is a "project(Project)" command on the first line. This warning is for project developers. Use -Wno-dev to suppress it.
[ 8%] Built target sha256
[ 8%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
[ 9%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as
cmake_minimum_required(VERSION 3.22)
should be added at the top of the file. The version specified may be lower if you wish to support older CMake versions for this project. For more information run "cmake --help-policy CMP0000". This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done -- Generating done -- Build files have been written to: /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build [ 9%] Performing build step for 'vulkan-shaders-gen' gmake[3]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule. [ 50%] Building CXX object CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.o [ 9%] Built target xxhash [100%] Linking CXX executable /home/ubuntu/llama.cpp/build-android/bin/vulkan-shaders-gen [100%] Built target vulkan-shaders-gen [ 9%] Built target vulkan-shaders-gen-build [ 9%] Performing install step for 'vulkan-shaders-gen' -- Install configuration: "Release" [ 9%] Built target vulkan-shaders-gen-install [ 10%] Completed 'vulkan-shaders-gen' [ 10%] Built target vulkan-shaders-gen [ 10%] Linking CXX shared library ../../bin/libggml-base.so [ 10%] Built target ggml-base [ 11%] Generate vulkan shaders ggml_vulkan: Generating and compiling shaders to SPIR-V cannot compile matmul_f32_f16_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_fp32.spv -DACC_TYPE=float -DB_TYPE=float16_t -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT_TYPE=float
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
cannot compile matmul_f32_f32_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f32_fp32.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=1
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
cannot compile cannot compile matmul_f32_f16_aligned_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_aligned_fp32.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=f16vec4 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=4 -DLOAD_VEC_B=4
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
cannot compile matmul_f16_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_fp32.spv -DACC_TYPE=float -DB_TYPE=float16_t -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT_TYPE=float
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
cannot compile matmul_f32_f32_aligned_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f32_aligned_fp32.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=vec4 -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=4 -DLOAD_VEC_B=4
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
matmul_f16_aligned_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_aligned_fp32.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=f16vec4 -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=4 -DLOAD_VEC_B=4
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
cannot compile matmul_f16_f32_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_f32_fp32.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=1
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
[ 12%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o [ 13%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o cannot compile matmul_f16_f32_aligned_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_f32_aligned_fp32.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=vec4 -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=4 -DLOAD_VEC_B=4
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
cannot compile matmul_q4_0_f32_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f32_fp32.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=2
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o [ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o [ 14%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o [ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o [ 14%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o cannot compile cannot compile cannot compile matmul_q4_1_f32_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f32_fp32.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=2
cannot compile cannot compile matmul_q4_0_f16_aligned_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_aligned_fp32.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=f16vec4 -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=2 -DLOAD_VEC_B=4
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found matmul_q4_0_f32_aligned_fp32matmul_q4_0_f16_fp32matmul_q4_1_f32_aligned_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_fp32.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=2
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f32_aligned_fp32.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=vec4 -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=2 -DLOAD_VEC_B=4
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f32_aligned_fp32.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=vec4 -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=2 -DLOAD_VEC_B=4
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
cannot compile matmul_q4_1_f16_aligned_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_aligned_fp32.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=f16vec4 -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=2 -DLOAD_VEC_B=4
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
cannot compile matmul_q4_1_f16_fp32
Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_fp32.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT_TYPE=float -DLOAD_VEC_A=2
sh: 1: Vulkan_GLSLC_EXECUTABLE-NOTFOUND: not found
[ 15%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o [ 15%] Linking CXX shared library ../../bin/libggml-cpu.so [ 15%] Built target ggml-cpu
and the compile will pause at "[ 15%] Built target ggml-cpu".
Please check if this branch fixes the glslc mismatch: https://github.com/jeffbolznv/llama.cpp/tree/glslc_mismatch.
-DVulkan_GLSLC_EXECUTABLE=/home/ubuntu/android-ndk-r28/shader-tools/linux-x86_64/glslc
Given how old it is, I kind of think it's a bad idea to force it to use the glslc from the ndk.
would it make sense to simply skip find_program(GLSLC_EXECUTABLE glslc) step if it's overridden instead of completely removing it?
This is so weird. If I remove -DVulkan_GLSLC_EXECUTABLE=, the compilation information is as follows
I don't understand why it wouldn't have picked up your the one from your Vulkan SDK. Was this a clean build (with no cmake caching)?
would it make sense to simply skip find_program(GLSLC_EXECUTABLE glslc) step if it's overridden instead of completely removing it?
I'm not sure if that line was even doing anything. It was setting GLSLC_EXECUTABLE, which is different from Vulkan_GLSLC_EXECUTABLE, and wasn't obviously used anywhere. I thought maybe there was some subtle funny business going on in cmake...
I don't understand why it wouldn't have picked up your the one from your Vulkan SDK. Was this a clean build (with no cmake caching)? Yes, all settings are clean. In order not to keep the cmake cache, I deleted and downloaded multiple glslc_mismatch/llama.cpp.
I'm not sure if that line was even doing anything. It was setting
GLSLC_EXECUTABLE, which is different fromVulkan_GLSLC_EXECUTABLE, and wasn't obviously used anywhere. I thought maybe there was some subtle funny business going on in cmake...
I checked ggml/src/ggml-vulkan/CMakeLists.txt and cmake did successfully find all Vulkan setting in the Android NDK except Vulkan_GLSLC_EXECUTABLE.
~/llama.cpp$ rm -rf build-android/ ~/llama.cpp$ cmake -DCMAKE_TOOLCHAIN_FILE=/home/ubuntu/android-ndk-r28/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-34 -DCMAKE_C_FLAGS="-march=armv8.7a" -DCMAKE_CXX_FLAGS="-march=armv8.7a" -DGGML_OPENMP=OFF -DGGML_LLAMAFILE=OFF -DGGML_VULKAN=ON -B build-android
-- Found Vulkan: /home/ubuntu/android-ndk-r28/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/34/libvulkan.so
-- Vulkan_FOUND: TRUE
-- Vulkan_INCLUDE_DIRS: /home/ubuntu/android-ndk-r28/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include
-- Vulkan_LIBRARIES: /home/ubuntu/android-ndk-r28/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/34/libvulkan.so
-- Vulkan_GLSLC_EXECUTABLE: Vulkan_GLSLC_EXECUTABLE-NOTFOUND
-- CMAKE_MODULE_PATH: /home/ubuntu/llama.cpp/cmake/
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 supported by glslc
-- Host compiler: /usr/bin/gcc /usr/bin/g++
-- vulkan-shaders-gen toolchain file: /home/ubuntu/llama.cpp/build-android/host-toolchain.cmake
-- Including Vulkan backend
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/llama.cpp/build-android
Hey everyone sorry for the delay jumping into this conversation. I wanted to let everyone know that there's a separate way to override the toolchain file using -DGGML_VULKAN_SHADERS_GEN_TOOLCHAIN=/path/to/toolchain.cmake. By default, when cross-compiling the ggml/src/ggml-vulkan/cmake/host-toolchain.cmake.in template is used to compile vulkan-shaders-gen.
I am wondering if modifying this, or supplying a custom toolchain which adds the proper NDK path works? The lines here force the search paths, which may be causing an issue:
From host-toolchain.cmake.in:
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE NEVER)
This is typically desired for a "standard" cross-compile where the system-supplied glslc is used, but in Android's case with multiple NDKs and so forth, there will be different paths to consider and so we probably need to supply two toolchains. If this is a common problem we could consider storing a general toolchain to supply for Android vulkan-shaders-gen builds.
@liuxiaotiao @barzoj We just merged in #12272 to fix cross-compiling with coopmat support—if you have some time to check whether it fixes this issue it would be greatly appreciated! Thanks.
@bandoti Using the following cmake command, vulkan cross compilation is successful without any further modifications to any cmake files. cmake \ -DCMAKE_TOOLCHAIN_FILE=/home/ubuntu/android-ndk-r28/build/cmake/android.toolchain.cmake \ -DANDROID_ABI=arm64-v8a \ -DANDROID_PLATFORM=android-33 \ -DCMAKE_C_FLAGS="-march=armv8.7a" \ -DCMAKE_CXX_FLAGS="-march=armv8.7a" \ -DGGML_OPENMP=OFF \ -DGGML_LLAMAFILE=OFF \ -DGGML_VULKAN=ON \ -DVulkan_INCLUDE_DIR=/home/ubuntu/Vulkan-Headers/include \ -DVulkan_LIBRARY=/home/ubuntu/android-ndk-r28/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/33/libvulkan.so \ -DVulkan_GLSLC_EXECUTABLE=/home/ubuntu/android-ndk-r28/shader-tools/linux-x86_64/glslc \ -B build-vulkan Thank you for your efforts.