LLamaSharp icon indicating copy to clipboard operation
LLamaSharp copied to clipboard

Information on new important updates in llama.cpp

Open zsogitbe opened this issue 10 months ago • 1 comments

There have been 3 important updates to the llama.cpp recently (a few days ago). One of them is the addition of pipeline parallelism (multi-threading) (https://github.com/ggerganov/llama.cpp/pull/6017). The other important update was the correction of the embeddings bug. And one more update to release all GPU memory (this was also a bug - memory was not released). Thank you slaren!

These are very important updates. Looking forward to the addition of the latest llama.cpp code to LLamaSharp.

zsogitbe avatar Mar 27 '24 07:03 zsogitbe

See #662, that update should include these things.

martindevans avatar Apr 14 '24 14:04 martindevans