nixpkgs
nixpkgs copied to clipboard
NixOS
Build failure: llama-cpp (with vulkanSupport = true)
Nixpkgs version
- Unstable (25.05)
Steps to reproduce
Run nix-shell -p "llama-cpp.override { vulkanSupport = true; }"
or
- Put this into
configuration.nix:
environment.systemPackages = with pkgs; [
(llama-cpp.override { vulkanSupport = true; })
];
- Run
sudo nixos-rebuild build --upgrade
Can Hydra reproduce this build failure?
No, Hydra cannot reproduce this build failure.
Link to Hydra build job
No response
Relevant log output
unpacking 1 channels...
building Nix...
building the system configuration...
these 13 derivations will be built:
/nix/store/dr00cv420gm1k1f2dxdibqcd1yjvr0x4-llama-cpp-5416.drv
/nix/store/7mf684bbi329lmvqphsk7m1b252qc23x-system-path.drv
/nix/store/9pm1b08s3a10fb9shbp8wdxsz5hpz9b0-dbus-1.drv
/nix/store/w0ixkpjf1604z5cwb0wlk9xg8nz0l3ic-X-Restart-Triggers-dbus.drv
/nix/store/mw68d0qfk5rspfa1bdw09ay8hzg53mjf-unit-dbus.service.drv
/nix/store/fbxdr0r3akblrpqgbzkq9723cxq4n5h6-X-Restart-Triggers-polkit.drv
/nix/store/nspv62684ypnbv2lw86cv3kwgfjmmhyy-unit-polkit.service.drv
/nix/store/p9mkwm9niw721p68vmvmxi7x7pakv1hw-unit-accounts-daemon.service.drv
/nix/store/3sa4gb2jkz0dm06fdy0c7hsz8crvwv2f-system-units.drv
/nix/store/3vdvlhd23bwfrid85asw6xggs4ljq53g-unit-dbus.service.drv
/nix/store/z0igvr5ykp74xwbvgvn30pbpcakshjg0-user-units.drv
/nix/store/wy7lzlvxa5q386hk23n0c2g6ahrffp54-etc.drv
/nix/store/8pml312zkyq965k3q370g1m9z5alqqjn-nixos-system-a-25.11pre803506.2795c506fe8f.drv
building '/nix/store/dr00cv420gm1k1f2dxdibqcd1yjvr0x4-llama-cpp-5416.drv'...
Running phase: unpackPhase
unpacking source archive /nix/store/ass2w6972dxwhqa2wqqsfjkrizi2siz4-source
source root is source
Running phase: patchPhase
Running phase: updateAutotoolsGnuConfigScriptsPhase
Running phase: configurePhase
fixing cmake files...
cmake flags: -GNinja -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DCMAKE_INSTALL_LOCALEDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/share/locale -DCMAKE_INSTALL_LIBEXECDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/libexec -DCMAKE_INSTALL_LIBDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/lib -DCMAKE_INSTALL_DOCDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/share/doc/llama-cpp -DCMAKE_INSTALL_INFODIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/share/info -DCMAKE_INSTALL_MANDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/share/man -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/include -DCMAKE_INSTALL_SBINDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/sbin -DCMAKE_INSTALL_BINDIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/bin -DCMAKE_INSTALL_NAME_DIR=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_STRIP=/nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/strip -DCMAKE_RANLIB=/nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/ranlib -DCMAKE_AR=/nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/ar -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=/nix/store/vvbn4wfcq59f0j3dw2rxi0cz20iz29k3-llama-cpp-5416 -DGGML_NATIVE:BOOL=FALSE -DLLAMA_BUILD_SERVER:BOOL=TRUE -DLLAMA_CURL:BOOL=TRUE -DBUILD_SHARED_LIBS:BOOL=TRUE -DGGML_BLAS:BOOL=FALSE -DGGML_CLBLAST:BOOL=FALSE -DGGML_CUDA:BOOL=FALSE -DGGML_HIP:BOOL=FALSE -DGGML_METAL:BOOL=FALSE -DGGML_RPC:BOOL=FALSE -DGGML_VULKAN:BOOL=TRUE
-- The C compiler identification is GNU 14.2.1
-- The CXX compiler identification is GNU 14.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /nix/store/805a5wv1cyah5awij184yfad1ksmbh9f-git-2.49.0/bin/git (found version "2.49.0")
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- x86 detected
-- Adding CPU backend variant ggml-cpu:
-- Found Vulkan: /nix/store/nszik1q8ffmvsqk54kbc75dwyxwvi2nm-vulkan-loader-1.4.313.0/lib/libvulkan.so (found version "1.4.313") found components: glslc missing components: glslangValidator
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 not supported by glslc
-- GL_EXT_integer_dot_product not supported by glslc
-- GL_EXT_bfloat16 not supported by glslc
-- Including Vulkan backend
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
CMake Warning at common/CMakeLists.txt:32 (message):
Git repository not found; to enable automatic generation of build info,
make sure Git is installed and the project is a Git repository.
-- Found CURL: /nix/store/v578vkzh0qhzczjvrzf64lqb2c74d5pk-curl-8.13.0/lib/libcurl.so (found version "8.13.0")
-- Configuring done (1.4s)
-- Generating done (0.1s)
CMake Warning:
Manually-specified variables were not used by the project:
CMAKE_EXPORT_NO_PACKAGE_REGISTRY
CMAKE_POLICY_DEFAULT_CMP0025
-- Build files have been written to: /build/source/build
cmake: enabled parallel building
cmake: enabled parallel installing
Running phase: buildPhase
build flags: -j32
[1/237] Creating directories for 'vulkan-shaders-gen'
[2/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o
[3/237] No download step for 'vulkan-shaders-gen'
[4/237] Generating build details from Git
-- Found Git: /nix/store/805a5wv1cyah5awij184yfad1ksmbh9f-git-2.49.0/bin/git (found version "2.49.0")
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
[5/237] No update step for 'vulkan-shaders-gen'
[6/237] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o
[7/237] No patch step for 'vulkan-shaders-gen'
[8/237] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o
[9/237] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[10/237] Building CXX object tools/mtmd/CMakeFiles/llama-minicpmv-cli.dir/deprecation-warning.cpp.o
[11/237] Building CXX object tools/mtmd/CMakeFiles/llama-gemma3-cli.dir/deprecation-warning.cpp.o
[12/237] Building CXX object tools/mtmd/CMakeFiles/llama-llava-cli.dir/deprecation-warning.cpp.o
[13/237] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o
[14/237] Building CXX object tools/mtmd/CMakeFiles/llama-qwen2vl-cli.dir/deprecation-warning.cpp.o
[15/237] Linking CXX executable bin/llama-llava-cli
[16/237] Linking CXX executable bin/llama-gemma3-cli
[17/237] Linking CXX executable bin/llama-minicpmv-cli
[18/237] Linking CXX executable bin/llama-qwen2vl-cli
[19/237] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[20/237] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o
[21/237] Performing configure step for 'vulkan-shaders-gen'
-- The C compiler identification is GNU 14.2.1
-- The CXX compiler identification is GNU 14.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /nix/store/0fsnicvfpf55nkza12cjnad0w84d6ba7-gcc-wrapper-14.2.1.20250322/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Enabling coopmat glslc support
-- Configuring done (0.9s)
-- Generating done (0.0s)
-- Build files have been written to: /build/source/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[22/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/vec.cpp.o
[23/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/llamafile/sgemm.cpp.o
[24/237] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[25/237] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[26/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o
[27/237] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[28/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[29/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[30/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
[31/237] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[32/237] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o
[33/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/binary-ops.cpp.o
[34/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o
[35/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/unary-ops.cpp.o
[36/237] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[37/237] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ops.cpp.o
[38/237] Performing build step for 'vulkan-shaders-gen'
[1/2] Building CXX object CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.cpp.o
[2/2] Linking CXX executable /build/source/build/bin/vulkan-shaders-gen
[39/237] Performing install step for 'vulkan-shaders-gen'
FAILED: ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/vulkan-shaders-gen-install /build/source/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/vulkan-shaders-gen-install
cd /build/source/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build && /nix/store/wphngc22a7aphbp5pi5jqmqlsqmisgn5-cmake-3.31.6/bin/cmake --install . && /nix/store/wphngc22a7aphbp5pi5jqmqlsqmisgn5-cmake-3.31.6/bin/cmake -E touch /build/source/build/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/vulkan-shaders-gen-install
-- Install configuration: ""
CMake Error at cmake_install.cmake:52 (file):
file INSTALL cannot find "/build/source/build/bin/vulkan-shaders-gen": No
such file or directory.
[40/237] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
ninja: build stopped: subcommand failed.
error: builder for '/nix/store/dr00cv420gm1k1f2dxdibqcd1yjvr0x4-llama-cpp-5416.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/7mf684bbi329lmvqphsk7m1b252qc23x-system-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/fbxdr0r3akblrpqgbzkq9723cxq4n5h6-X-Restart-Triggers-polkit.drv' failed to build
error: 1 dependencies of derivation '/nix/store/9pm1b08s3a10fb9shbp8wdxsz5hpz9b0-dbus-1.drv' failed to build
error: 1 dependencies of derivation '/nix/store/p9mkwm9niw721p68vmvmxi7x7pakv1hw-unit-accounts-daemon.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/w0ixkpjf1604z5cwb0wlk9xg8nz0l3ic-X-Restart-Triggers-dbus.drv' failed to build
error: 1 dependencies of derivation '/nix/store/nspv62684ypnbv2lw86cv3kwgfjmmhyy-unit-polkit.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/3vdvlhd23bwfrid85asw6xggs4ljq53g-unit-dbus.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/mw68d0qfk5rspfa1bdw09ay8hzg53mjf-unit-dbus.service.drv' failed to build
error: 3 dependencies of derivation '/nix/store/3sa4gb2jkz0dm06fdy0c7hsz8crvwv2f-system-units.drv' failed to build
error: 1 dependencies of derivation '/nix/store/z0igvr5ykp74xwbvgvn30pbpcakshjg0-user-units.drv' failed to build
error: 4 dependencies of derivation '/nix/store/wy7lzlvxa5q386hk23n0c2g6ahrffp54-etc.drv' failed to build
error: 2 dependencies of derivation '/nix/store/8pml312zkyq965k3q370g1m9z5alqqjn-nixos-system-a-25.11pre803506.2795c506fe8f.drv' failed to build
Additional context
This worked up until at least April 29.
Without vulkanSupport = true it succeeds.
System metadata
- system:
"x86_64-linux" - host os:
Linux 6.14.3, NixOS, 25.05 (Warbler), 25.05pre789333.f771eb401a46 - multi-user?:
yes - sandbox:
yes - version:
nix-env (Nix) 2.28.2 - channels(root):
"nixos" - nixpkgs:
/nix/store/nvw9cf1nffldxzahsbiv7jgfp832i26b-nixos/nixos
Notify maintainers
@dit7ya @elohmeier @philiptaron @xddxdd
Note for maintainers: Please tag this issue in your pull request description. (i.e. Resolves #ISSUE.)
I assert that this issue is relevant for Nixpkgs
- [x] I assert that this is a bug and not a support request.
- [x] I assert that this is not a duplicate of an existing issue.
- [x] I assert that I have read the NixOS Code of Conduct and agree to abide by it.
Is this issue important to you?
Add a :+1: reaction to issues you find important.
if it helps anyone, the ustream commit causing this is https://github.com/ggml-org/llama.cpp/commit/09d13d94fb169093095416ac6323551c37b75339 (https://github.com/ggml-org/llama.cpp/pull/13263)
I won't be able to work on this, but I will gladly test and merge PRs that do.
With this overlay I'm able to build and run llama.cpp
(final: prev: {
llama-cpp = prev.llama-cpp.overrideAttrs (oldAttrs: {
postPatch =
(oldAttrs.postPatch or "")
+ ''
echo "Applying patch to ggml/src/ggml-vulkan/CMakeLists.txt"
sed -i '/DCMAKE_RUNTIME_OUTPUT_DIRECTORY/d' ggml/src/ggml-vulkan/CMakeLists.txt
'';
});
})
Is anyone here using both Nixpkgs llama.cpp and Windows with a development toolchain set up? I think the upstream PR needs fixing the Windows side…
I tried updating nixpkgs to see if the latest commit with a merged PR solved it, but I ran into this:
error: Cannot build '/nix/store/l7q2dhq28qxdj5ccn3qa0rmvcd64gcym-llama-cpp-5702.drv'.
Reason: builder failed with exit code 1.
Output paths:
/nix/store/cwhl7ha22pi4cj6bskj9dlgagamq6gai-llama-cpp-5702
Last 25 log lines:
> | ^~~~~~
> /build/source/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2110:48: error: 'matmul_id_bf16_aligned_cm2_len' was not declared in this scope; did you mean 'matmul_id_f16_aligned_cm2_len'?
> 2110 | CREATE_MM(pipeline_matmul_id_bf16, matmul_id_bf16, , wg_denoms, warptile, vk_mat_mat_id_push_constants, 4)
> | ^~~~~~~~~~~~~~
> /build/source/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2072:101: note: in definition of macro 'CREATE_MM'
> 2072 | ggml_vk_create_pipeline(device, device-> PIPELINE_NAME ->a_l, #NAMELC #F16ACC "_aligned_l", NAMELC ## _aligned ## F16ACC ## _cm2_len, NAMELC ## _aligned ## F16ACC ## _cm2_data, "main", PARAMCOUNT, sizeof(PUSHCONST), l_ ## WG_DENOMS, l_ ## WARPTILE, l_align); \
> | ^~~~~~
> /build/source/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2110:48: error: 'matmul_id_bf16_aligned_cm2_data' was not declared in this scope; did you mean 'matmul_id_f16_aligned_cm2_data'?
> 2110 | CREATE_MM(pipeline_matmul_id_bf16, matmul_id_bf16, , wg_denoms, warptile, vk_mat_mat_id_push_constants, 4)
> | ^~~~~~~~~~~~~~
> /build/source/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2072:143: note: in definition of macro 'CREATE_MM'
> 2072 | ggml_vk_create_pipeline(device, device-> PIPELINE_NAME ->a_l, #NAMELC #F16ACC "_aligned_l", NAMELC ## _aligned ## F16ACC ## _cm2_len, NAMELC ## _aligned ## F16ACC ## _cm2_data, "main", PARAMCOUNT, sizeof(PUSHCONST), l_ ## WG_DENOMS, l_ ## WARPTILE, l_align); \
> |
Adding this patch gets llama-cpp building again for me with Vulkan support.
From 654575ed77996923556940b8d6a7027c4f890436 Mon Sep 17 00:00:00 2001
From: Quag <[email protected]>
Date: Sat, 28 Jun 2025 13:40:24 -0700
Subject: [PATCH] Remove GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT
---
ggml/src/ggml-vulkan/CMakeLists.txt | 6 ---
ggml/src/ggml-vulkan/ggml-vulkan.cpp | 44 -------------------
.../ggml-vulkan/vulkan-shaders/CMakeLists.txt | 4 --
.../vulkan-shaders/vulkan-shaders-gen.cpp | 2 -
4 files changed, 56 deletions(-)
diff --git a/ggml/src/ggml-vulkan/CMakeLists.txt b/ggml/src/ggml-vulkan/CMakeLists.txt
index b97e7bf9..9cdc62a4 100644
--- a/ggml/src/ggml-vulkan/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/CMakeLists.txt
@@ -70,12 +70,6 @@ if (Vulkan_FOUND)
"GGML_VULKAN_INTEGER_DOT_GLSLC_SUPPORT"
)
- test_shader_extension_support(
- "GL_EXT_bfloat16"
- "${CMAKE_CURRENT_SOURCE_DIR}/vulkan-shaders/test_bfloat16_support.comp"
- "GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT"
- )
-
target_link_libraries(ggml-vulkan PRIVATE Vulkan::Vulkan)
target_include_directories(ggml-vulkan PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
diff --git a/ggml/src/ggml-vulkan/ggml-vulkan.cpp b/ggml/src/ggml-vulkan/ggml-vulkan.cpp
index 996ccbf6..93167688 100644
--- a/ggml/src/ggml-vulkan/ggml-vulkan.cpp
+++ b/ggml/src/ggml-vulkan/ggml-vulkan.cpp
@@ -2095,11 +2095,6 @@ static void ggml_vk_load_shaders(vk_device& device) {
CREATE_MM(PIPELINE_NAME . f32acc, NAMELC, , WG_DENOMS, WARPTILE, PUSHCONST, PARAMCOUNT) \
CREATE_MM2(pipeline_matmul_f16, matmul_f16, wg_denoms, warptile, vk_mat_mat_push_constants, 3)
-#if defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
- if (device->coopmat_bf16_support) {
- CREATE_MM(pipeline_matmul_bf16, matmul_bf16, , wg_denoms, warptile, vk_mat_mat_push_constants, 3)
- }
-#endif
CREATE_MM2(pipeline_dequant_mul_mat_mat_f16[GGML_TYPE_Q4_0], matmul_q4_0_f16, mmq_wg_denoms, warptile_mmq, vk_mat_mat_push_constants, 3)
CREATE_MM2(pipeline_dequant_mul_mat_mat_f16[GGML_TYPE_Q4_1], matmul_q4_1_f16, mmq_wg_denoms, warptile_mmq, vk_mat_mat_push_constants, 3)
CREATE_MM2(pipeline_dequant_mul_mat_mat_f16[GGML_TYPE_Q5_0], matmul_q5_0_f16, mmq_wg_denoms, warptile_mmq, vk_mat_mat_push_constants, 3)
@@ -2121,11 +2116,6 @@ static void ggml_vk_load_shaders(vk_device& device) {
CREATE_MM2(pipeline_dequant_mul_mat_mat_f16[GGML_TYPE_IQ4_NL], matmul_iq4_nl_f16, mmq_wg_denoms, warptile_mmq, vk_mat_mat_push_constants, 3)
CREATE_MM2(pipeline_matmul_id_f16, matmul_id_f16, wg_denoms, warptile, vk_mat_mat_id_push_constants, 4)
-#if defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
- if (device->coopmat_bf16_support) {
- CREATE_MM(pipeline_matmul_id_bf16, matmul_id_bf16, , wg_denoms, warptile, vk_mat_mat_id_push_constants, 4)
- }
-#endif
CREATE_MM(pipeline_dequant_mul_mat_mat_id[GGML_TYPE_Q4_0].f16acc, matmul_id_q4_0_f16, , mmqid_wg_denoms, warptile_mmqid, vk_mat_mat_id_push_constants, 4)
CREATE_MM(pipeline_dequant_mul_mat_mat_id[GGML_TYPE_Q4_1].f16acc, matmul_id_q4_1_f16, , mmqid_wg_denoms, warptile_mmqid, vk_mat_mat_id_push_constants, 4)
CREATE_MM(pipeline_dequant_mul_mat_mat_id[GGML_TYPE_Q5_0].f16acc, matmul_id_q5_0_f16, , mmqid_wg_denoms, warptile_mmqid, vk_mat_mat_id_push_constants, 4)
@@ -2179,11 +2169,6 @@ static void ggml_vk_load_shaders(vk_device& device) {
CREATE_MM(GGML_TYPE_F32, pipeline_matmul_f32_f16, matmul_f32_f16, , wg_denoms, warptile, vk_mat_mat_push_constants, 3, );
CREATE_MM2(GGML_TYPE_F16, pipeline_matmul_f16, matmul_f16, wg_denoms, warptile, vk_mat_mat_push_constants, 3, );
CREATE_MM2(GGML_TYPE_F16, pipeline_matmul_f16_f32, matmul_f16_f32, wg_denoms, warptile, vk_mat_mat_push_constants, 3, );
-#if defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
- if (device->coopmat_bf16_support) {
- CREATE_MM(GGML_TYPE_BF16, pipeline_matmul_bf16, matmul_bf16, , wg_denoms, warptile, vk_mat_mat_push_constants, 3, )
- }
-#endif
if (device->coopmat_acc_f16_support) {
CREATE_MM2(GGML_TYPE_Q4_0, pipeline_dequant_mul_mat_mat[GGML_TYPE_Q4_0], matmul_q4_0_f32, mmq_wg_denoms, warptile_mmq, vk_mat_mat_push_constants, 3, );
@@ -2232,11 +2217,6 @@ static void ggml_vk_load_shaders(vk_device& device) {
CREATE_MM(GGML_TYPE_F32, pipeline_matmul_id_f32, matmul_id_f32_f32, , wg_denoms, warptile, vk_mat_mat_push_constants, 4, _id);
CREATE_MM2(GGML_TYPE_F16, pipeline_matmul_id_f16, matmul_id_f16, wg_denoms, warptile, vk_mat_mat_push_constants, 4, _id);
CREATE_MM2(GGML_TYPE_F16, pipeline_matmul_id_f16_f32, matmul_id_f16_f32, wg_denoms, warptile, vk_mat_mat_push_constants, 4, _id);
-#if defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
- if (device->coopmat_bf16_support) {
- CREATE_MM(GGML_TYPE_BF16, pipeline_matmul_id_bf16, matmul_id_bf16, , wg_denoms, warptile, vk_mat_mat_push_constants, 4, _id);
- }
-#endif
if (device->coopmat_acc_f16_support) {
CREATE_MM(GGML_TYPE_Q4_0, pipeline_dequant_mul_mat_mat_id[GGML_TYPE_Q4_0].f16acc, matmul_id_q4_0_f32, _f16acc, mmq_wg_denoms, warptile_mmq, vk_mat_mat_id_push_constants, 4, _id);
@@ -2892,11 +2872,6 @@ static vk_device ggml_vk_get_device(size_t idx) {
} else if (strcmp("VK_KHR_shader_integer_dot_product", properties.extensionName) == 0 &&
!getenv("GGML_VK_DISABLE_INTEGER_DOT_PRODUCT")) {
device->integer_dot_product = true;
-#endif
-#if defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
- } else if (strcmp("VK_KHR_shader_bfloat16", properties.extensionName) == 0 &&
- !getenv("GGML_VK_DISABLE_BFLOAT16")) {
- bfloat16_support = true;
#endif
}
}
@@ -3306,25 +3281,6 @@ static vk_device ggml_vk_get_device(size_t idx) {
device->coopmat_int_n = prop.NSize;
device->coopmat_int_k = prop.KSize;
}
-#if defined(VK_KHR_shader_bfloat16) && defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
- if (prop.AType == VK_COMPONENT_TYPE_BFLOAT16_KHR &&
- prop.BType == VK_COMPONENT_TYPE_BFLOAT16_KHR &&
- prop.CType == VK_COMPONENT_TYPE_FLOAT32_KHR &&
- prop.ResultType == VK_COMPONENT_TYPE_FLOAT32_KHR &&
- (vk::ScopeKHR)prop.scope == vk::ScopeKHR::eSubgroup
- ) {
- // coopmat sizes not set yet
- if (device->coopmat_m == 0) {
- device->coopmat_bf16_support = true;
- device->coopmat_m = prop.MSize;
- device->coopmat_n = prop.NSize;
- device->coopmat_k = prop.KSize;
- } else if (device->coopmat_m == prop.MSize && device->coopmat_n == prop.NSize && device->coopmat_k == prop.KSize) {
- // Only enable if shape is identical
- device->coopmat_bf16_support = true;
- }
- }
-#endif
}
if (device->coopmat_m == 0 || !device->coopmat_acc_f32_support) {
diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
index e1f613fb..3c61a19b 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
@@ -15,10 +15,6 @@ if (GGML_VULKAN_INTEGER_DOT_GLSLC_SUPPORT)
add_compile_definitions(GGML_VULKAN_INTEGER_DOT_GLSLC_SUPPORT)
message(STATUS "Enabling dot glslc support")
endif()
-if (GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
- add_compile_definitions(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
- message(STATUS "Enabling bfloat16 glslc support")
-endif()
if (GGML_VULKAN_SHADER_DEBUG_INFO)
add_compile_definitions(GGML_VULKAN_SHADER_DEBUG_INFO)
message(STATUS "Enabling shader debug info")
diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp b/ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp
index c63345ec..c9252203 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/vulkan-shaders-gen.cpp
@@ -349,9 +349,7 @@ void matmul_shaders(bool fp16, bool matmul_id, bool coopmat, bool coopmat2, bool
std::string to_float_type = (coopmat || coopmat2) ? "uintBitsToBFloat16EXT" : "bf16_to_fp32";
// If bfloat16 is not supported, then only compile the scalar (promote to fp32) shader
-#if !defined(GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT)
if (!(coopmat || coopmat2))
-#endif
{
string_to_spv(shader_name + "_bf16_aligned", source_name, merge_maps(base_dict, {{"FLOAT_TYPE", FLOAT_TYPE("bf16")}, {"TO_FLOAT_TYPE", to_float_type}, {"DATA_A_BF16", "1"}, {"LOAD_VEC_A", load_vec_a}, {"LOAD_VEC_B", "4"}, {"B_TYPE", coopmat2 ? "bfloat16_t" : "u16vec4"}, {"D_TYPE", "float"}, {"B_IS_FLOAT", "1"}, {"ALIGNED", "1"}}), fp16, coopmat, coopmat2, f16acc);
string_to_spv(shader_name + "_bf16", source_name, merge_maps(base_dict, {{"FLOAT_TYPE", FLOAT_TYPE("bf16")}, {"TO_FLOAT_TYPE", to_float_type}, {"DATA_A_BF16", "1"}, {"LOAD_VEC_A", load_vec_a_unaligned}, {"B_TYPE", coopmat2 ? "bfloat16_t" : "uint16_t"}, {"D_TYPE", "float"}, {"B_IS_FLOAT", "1"}}), fp16, coopmat, coopmat2, f16acc);
--
2.49.0
As an alternative the official repo has a flake.nix. Just set it in your inputs like this:
llama-cpp = {
url = "github:ggml-org/llama.cpp/b5896";
};
Then in your environment.systemPakages add:
inputs.llama-cpp.packages.x86_64-linux.vulkan
EDIT: Wasn't able to build the latest version but this one: b5896 worked for me.
That is still an issue.
Also, I think it would have been great to have a separate llama-cpp-vulkan package to avoid building it on lower-end devices (like SBCs), but that is a completely different discussion.
Right now, I suppose we need #422031 to get to unstable? What shaderc version do you get on the checkout you use?
@7c6f434c I cannot test it right now, but your suggestion should indeed resolve the issue. I am using an older unstable branch right now (from weeks ago).
Today's might still not be fresh enough, but as the staging cycle converges, this change will eventually propagate
@7c6f434c Cool, thanks!
For now it seems that just the following is enough to fix the build:
index b97e7bf9..9cdc62a4 100644
--- a/ggml/src/ggml-vulkan/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/CMakeLists.txt
@@ -70,12 +70,6 @@ if (Vulkan_FOUND)
"GGML_VULKAN_INTEGER_DOT_GLSLC_SUPPORT"
)
- test_shader_extension_support(
- "GL_EXT_bfloat16"
- "${CMAKE_CURRENT_SOURCE_DIR}/vulkan-shaders/test_bfloat16_support.comp"
- "GGML_VULKAN_BFLOAT16_GLSLC_SUPPORT"
- )
-
target_link_libraries(ggml-vulkan PRIVATE Vulkan::Vulkan)
target_include_directories(ggml-vulkan PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
And this is what I have in overlays:
# llama.cpp with vulkan support
(self: super: {
llama-cpp-vulkan = (super.llama-cpp.override {
vulkanSupport = true;
}).overrideAttrs (oldAttrs: {
patches = (oldAttrs.patches or [ ]) ++ [
./nixpkgs-overlays/llama-cpp-vulkan.patch
];
});
})
A shame to see this go stale for two weeks, I've left an upvote and encourage anyone with an AMD GPU trying to use llama.cpp with Vulkan to do the same. Do let me know if there's any tutorials out there to test out the PRs and patches of this thread, I might give it a shot and see what I get with my RX 7900 XT.
