Hendrik

Same problem after a restart. Creating new instances and starting them works, previous instances don't. Also macOS 14.5 and M1 Max.

Thanks for clarifying!

As I have looked around in the llama.cpp-project, I found [this](https://github.com/ggerganov/llama.cpp/blob/master/llama.h#L227) callback. A sample usage can be found [here](https://github.com/ggerganov/llama.cpp/blob/81d41628f90c4a2e68e6253f20cd3c46235f41a5/examples/simple/simple.cpp#L9-L34) (but the `int node_index` must be removed) and a more extensive...

Oh wow, this looks great! Thank you! I will see how far I can get with it 😄

Hi @reuank, I switched from llama-cpp-python to llama.cpp for other reasons, and started implementing an attention score collecting callback for the server-implementation. I'm not sure how / if this may...