LLamaSharp
LLamaSharp copied to clipboard
Setting up a non-development runtime environment as a published .net application.
Description
Greetings everyone, I’ve done about a good six months of development diving into LlamaSharp including Microsoft's Kernel for rag. | validated my App works on two separate dev boxes using CUDA, CPU and Vulkan ATI but the challenge I am having is moving my application to a nondev machine via a published application. I got .Net core 8.0 runtime and SDK installed and upon running my binary | get an error stating that I don’t have a CPU backend installed... reading around the LlamaCPP and LlamaSharp(here) GitHub issues I didn't spot anything that stuck out when it came to the deployment of applications outside of a dev environment but I did see a few post recommending OpenBlas or CLBLAS as a CPU backend that involved recompiling LlamaCPP which I am not sure if that’s actually required for a LlamaSharp deployment but I can’t say for certain because on my dev box ljust run install-Package Llama.backend.cpu and everything works from there and I’m not sure if that downloads and configures OpenBlas behind the scenes. Can anyone confirm if I am going down the correct road or if I should be looking into something else? Any referenced links are also appreciated. I think my main struggle is finding detailed documentation on setting up a runtime environment vs. a dev environment. Thank you everyone in advanced!