LLamaSharp
LLamaSharp copied to clipboard
SciSharp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
I had a go at using Kernel Memory to store documents (1.3MB), but it kept crashing after `memory.SaveReferenceAsync`, or with the latest LLamaSharp commit and Kernel Memory LLamaSharp connector, `GenerationMemory.ImportDocumentAsync`....
llama_new_context_with_model: n_ctx = 7168 llama_new_context_with_model: freq_base = 10000.0 llama_new_context_with_model: freq_scale = 1 llama_kv_cache_init: offloading v cache to GPU llama_kv_cache_init: offloading k cache to GPU llama_kv_cache_init: VRAM kv self = 8680.00...
I am using the grammar below with [OpenHermes-2.5-Mistral-7B-16k-GGUF](https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-16k-GGUF). ``` root ::= (object | array) endline? endline ::= "" ws value ::= object | array | string | number | ("true"...
I've been trying to create a chatbot with the ChatSession class and IHistoryTransform implementation to use the correct template for the LLM I am testing LLamaSharp with. I ran into...
related issue: #345
I'm setting up various inferences backends in my project, including LLamaSharp. I wanted to keep a unified history across these backends, so I created a custom structure to handle it....
I have cloned the repo and when i run the ChatSessionStripRoleName example it doesn't reliably work, sometimes the role name still comes through in the output, it is my understanding...
I am running in osx on an M1 and have been able to run the Web project without any issues. The problem arises when it's run within a container. I...
I tried to build my own `libllama.dll` (at [6b73ef120114beb5664ea94aab48d07ed248ee52](https://github.com/ggerganov/llama.cpp/commit/6b73ef120114beb5664ea94aab48d07ed248ee52), since that's what the committed `libllama.dll` is built from) ```powershell cmake --build . --target llama --config Debug copy "C:\Source\llama.cpp\build\bin\Debug\llama.dll" "C:\Source\LLamaSharp\LLama.Examples\bin\Debug\net7.0\libllama.dll" ```...
Hi, How can i **fine tune** or **train** a llama model with LLamaSharp? I couldn't find a documentation about fine-tuning, training a model or using LLamaSharp Library for any other...
