pathwaycom
[Question]: could i use my local embedding model ?
Steps to reproduce
i have my own embedding encoder, and just request the http url to get vectors, then how to realize it and i also have my own LLM , and also i ask it just through the URL with ChatOpenai or OpenAI func , then how to replace the default Openai ? thx
Relevant log output
no
What did you expect to happen?
replace the embedding encoder and the LLM
Version
no
Docker Versions (if used)
No response
OS
Linux
On which CPU architecture did you run Pathway?
ARM64 (AArch64, Apple silicon)
@ucas010 please take a look at this guide: https://pathway.com/developers/templates/private-rag-ollama-mistral#_3-embedding-model-selection. It describes both selection of local embeddings, and of a connection to a custom LLM via LiteLLM wrapper.
The code described comes from this template https://github.com/pathwaycom/llm-app/tree/main/examples/pipelines/private-rag but can be used with any template.
For relevant documentation, see: https://pathway.com/developers/api-docs/pathway-xpacks-llm/llms https://pathway.com/developers/api-docs/pathway-xpacks-llm/embedders
thx, if i do not use pw vector store , how to deal with it ? or i just use faiss ? @dxtrous
I think there is no problem here: pw Vector Store is a built-in vector index in-memory (it is built around Tantivy, at similar performance levels as the FAISS implementation). You do not need any extra steps to set up pw Vector Store, it will work out of the box. Try running the code.
Setting up an external vector store or integrating with FAISS is significantly more work. We currently do not provide templates for this.
Hi @ucas010 did you give the built-in vector store a try? If we should close the issue, let me know.