llamafile icon indicating copy to clipboard operation
llamafile copied to clipboard

Fix 2 embeddings-related issues in server.cpp

Open k8si opened this issue 10 months ago • 0 comments

Misc embeddings fixes:

  • Fix #303 - /embeddings server endpoint returns actual embeddings rather than 0-vector
  • Fix #322 - Pull in upstream change from llama.cpp to improve locale handling during lowercase. (This was what was causing an error when using a BERT model with the /embeddings endpoint). Now efficient sentence embedding models like all-MiniLM-L6-v2 should work without a problem.

TODO:

  • [ ] Still need to test calls to the /embeddings endpoint with a batch of texts, thus far I have only tested with single texts (will finish this on Monday)

k8si avatar Apr 05 '24 22:04 k8si