node-llama-cpp
node-llama-cpp copied to clipboard
withcatai
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
### Issue description Electron crashes when loadModel finishes loading (beta) ### Expected Behavior After loading a model using this code and trying to create a context, I'd expect there not...
### Feature Description Is it possible for `node-llama-cpp` to support React Native projects? I know that this project depends on many node standard libraries, but is it possible to make...
### Issue description When I use node-llama-cpp to run inference, cloudrun fails with a 503 error. ### Expected Behavior Run inference in cloudrun without any issues. ### Actual Behavior I...
### Issue description Failed to build with the latest version of Cuda. When using prebuilt binaries it says incompatible. ### Expected Behavior No errors with invoking `getLlama` ### Actual Behavior...
### What was unclear or otherwise insufficient? Downloading and running .exe under this section:  Cause the error on run:  ### Recommended Fix Fix the .exe file ### Additional...
Will be added once `llama.cpp`'s TTS APIs are finalized and [Orpheus](https://github.com/canopyai/Orpheus-TTS) ([issue](https://github.com/ggml-org/llama.cpp/issues/12476)) or [SparkTTS](https://github.com/SparkAudio/Spark-TTS) ([issue](https://github.com/ggml-org/llama.cpp/issues/12495)) models are supported.
### Issue description `Qwen3-Embedding-8B-Q4_K_M.gguf` (downloaded from huggingface) works on `gpu: false` (cpu) but not when run on vulkan. ### Expected Behavior `await embedContext.getEmbeddingFor(...)` returns successfully, or throws an error that...
### Feature Description _First of all, huge thanks for developing this addon. And I try to not disturb anyone by asking for features. This is literally my very first Github...
### Feature Description I can't reproduce my existing AMD hipBLAS build: ``` export ROCM_PATH=/usr/lib64/rocm export ROCM_DEVICE_LIB_PATH=$ROCM_PATH/llvm/lib/clang/18/amdgcn/bitcode cmake -S . -B build \ -DGGML_HIP=ON \ -DAMDGPU_TARGETS=gfx1030 \ -DCMAKE_BUILD_TYPE=Release \ -DCMAKE_C_COMPILER=hipcc \...
### Feature Description current on windows platform, cuda and vulkan are supported. is there a roadmap to support sycl? I think it will improve the performance especially on intel's gpu....
