[BUG]: RAG with KernelMemory -

[biapar]

Description

I've this error: llama_get_logits_ith: invalid logits id 214, reason: no logits [1] 16721 segmentation fault

Reproduction Steps

I follow this example https://scisharp.github.io/LLamaSharp/0.11.2/Examples/KernelMemorySaveAndLoad/ and use I use this model Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf

Environment & Configuration

• Operating system: MacOS Sequoia
• .NET runtime version: 8
• LLamaSharp version: 0.19
• CUDA version (if you are using cuda backend):
• CPU & GPU device: Apple M2 MAX

Known Workarounds

No response

[zsogitbe]

Is this the Embeddings = true problem in the model parameters, do you need to set it to false?

[freefer]

Do not use WithLLamaSharpDefaults as it will enable Embeddings causing exceptions, it can be used directly as an alternative WithLLamaSharpTextEmbeddingGeneration(new LLamaSharpTextEmbeddingGenerator(lsConfig, embWeights)) .WithLLamaSharpTextGeneration(new LlamaSharpTextGenerator(textWeights, context, executor, lsConfig.DefaultInferenceParams))

[biapar]

Have you a full example? thx

[github-actions[bot]]

This issue has been automatically marked as stale due to inactivity. If no further activity occurs, it will be closed in 7 days.