local.ai
local.ai copied to clipboard
louisgv
BUG | Can't run 0.5.1 on Windows, asks for additional dlls
Hi, the latest version 0.5.1 on Windows asks for additional dlls: cublas64_12.dll and cudart64_12.dll. Any requirements to install manually? Thanks!
Hmmm... I wonder if these's a way to pack these with the installation, or do we have to add them as additional asset...
@ke1ne you can grab those DLL from https://developer.nvidia.com/cuda-12-1-0-download-archive, but we should figure out how to bundle them
@louisgv After the CUDA Runtime installation, it didn't handle dlls until I added their directory (c:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2\bin) to the PATH environment variable. It seems to work now, thanks!
@ke1ne could you help me with reproducing the build DLL in your local machine? I don't have a Windows box handy atm, and the one I have nearby doesn't have a GPU xD......
Basically we need to include the DLL into the resources specification in #62
https://github.com/bytecodealliance/wasmtime/blob/c19c729214e2237902eb177609643cb6523b7f2b/cranelift/native/src/lib.rs#L184
@louisgv I will try to help for sure. Please instruct me what to do and what we need to achieve.
@ke1ne awesome! Can you try cloning this repo, checking out the 61-bug-cuda-dlls branch and see if you can add those dll into the build directory, compile and see if it built and run in dev mode? LMK if you need steps-by-steps instruction!
@louisgv You could probably use the build.rs of your tauri project to copy the correct dlls/so files to an '\dlls' folder in your build target which you can then always include via an "resources": [dlls/*] definition in the tauri.conf.json. This would make building very simple and you don't need to change anything in your workflows.
You should move it back until https://github.com/rustformers/llm/pull/325 gets mergen, then i can take a look at including the dlls.
FYI - might also not be allowed to distribute them, assuming they are shipped by the Nvidia driver normally.
