Martin Evans

`command buffer 0 failed with status 5` seems to indicate an out-of-memory error (ref: https://github.com/ggerganov/llama.cpp/issues/2048).

Unfortunately I don't think there's any way we can handle a GGML_ASSERT. It's defined [here](https://github.com/ggerganov/llama.cpp/blob/23b5e12eb5a76489b4c3ee22213a081da68b1809/ggml.h#L244) to call `abort()` which is about as fatal as it gets!

Yep that would be the only way to handle it (an `abort()` just destroys the process, with no way to recover). That's not something LLamaSharp does internally at the moment...

I can ask if you'd prefer not to, but LLamaSharp doesn't have any special pull in the llama.cpp project. To be honest at the moment I suspect any such request...

I've opened up https://github.com/ggerganov/llama.cpp/issues/4385

Although I will say I wouldn't expect this to change quickly if at all! It woud be a large change in both LLamaSharp and llama.cpp! If it's an issue you...

Some interesting discussion related to error handling in llama.cpp here: https://github.com/ggerganov/ggml/pull/701

I believe you can do something like this: ```csharp NativeApi.llama_log_set((level, message) => { // This will be called when llama.cpp wants to log a message. Do whatever you like! Console.WriteLine($"[{level}]:...

> we'll fix it soon. What's the planned fix here? As far as I'm aware we want to create a new context every time, so that it's truly stateless. Or...

To help debug this, I'd suggest trying to add something into [here](https://github.com/SciSharp/LLamaSharp/blob/267523e22a4df8485d16023a7b58d83314c6a046/LLama/Native/SafeLLamaContextHandle.cs#L217) which logs out the value of `ret` if it is non-null. Unfortunately llama.cpp doesn't return very detailed error...