LLamaSharp
LLamaSharp copied to clipboard
SciSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
### Description **IMPORTANT** Claude Code helped write the issue summary, so please forgive any factual inaccuracies that I couldn't personally verify. I couldn't find a duplicate bug other than issue...
Hello, First, of all, thank you for providing such useful tool. I am using LlamaSharp in a unity project and get stuck at the first call of the library at...
[BUG]: Error in version 0.25.0 - LLama.Exceptions.RuntimeError: Failed to load the native library.
### Description After upgrading to version 0.25.0, an error occurs when downloading: ``` LLama.Exceptions.RuntimeError: Failed to load the native library. Please check the log for more information. at LLama.Native.NativeLibraryUtils.TryLoadLibrary(NativeLibraryConfig config,...
``` using LLama; using LLama.Batched; using LLama.Common; using LLama.Native; using LLama.Sampling; using Spectre.Console; using System.Text; var ModelPath = Path.Combine(Environment.CurrentDirectory, "llava-v1.6-mistral-7b-Q5_K_S.gguf"); // change it to your own model path var ModelPath4...
### Background & Description With some new LLMs, there are new features called **tool calling**, such as with [Qwen 4B 2507 Instruct](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507) with Qwen Agent. I would like **LlamaSharp** to...
Fixes #382: On Windows, runtime files from backend packages are currently flattened during copy. This means that all files from subfolders end up in the same output directory, causing conflicts...
### Description I am trying to use multimodal model output, and the code is as follows, but it seems that the returns are all garbled. Are there any related use...
### Description When creating embeddings via the Microsoft.Extensions.AI IEmbeddingGenerator interface the underlying LLama.Native.SafeLLamaContextHandle throws an object disposed exception. Which makes a bit of sense when you look through how the...
### Description Please keep in mind that embeddings for chunks are created many times for a single document. You wrote the code so that the context is constantly being recreated....
The tokens list cannot be changed, it is the input list of tokens! Error line: https://github.com/SciSharp/LLamaSharp/blob/9fe066dffa8aeb53d47bdbd9d8d6e97988055b14/LLama/LLamaStatelessExecutor.cs#L137 Llama.cpp source: https://github.com/ggml-org/llama.cpp/blob/60325fa56f61c228464c9f065db3aa6a61f2156e/examples/main/main.cpp#L334 https://github.com/ggml-org/llama.cpp/blob/60325fa56f61c228464c9f065db3aa6a61f2156e/examples/main/main.cpp#L542 I'm trying to fix and rewrite it correctly, but so...