Nicola Procopio comments

Results 28 comments of


                                            Nicola Procopio

Create new VectorStore collections whenever the embedder changes

Thanks @pieroit , I was able to recreate the error it was giving me before the changes. ``` {"error":"Wrong input: Vector inserting error: expected dim: 1536, got 4096"} ``` Starting...

Create new VectorStore collections whenever the embedder changes

Hi @pieroit , I tried to split the problem: 1. configure the embedding dimension dinamically 2. create new VectorStore collections whenever the embedder changes At the moment, for point 1,...

Create new VectorStore collections whenever the embedder changes

@pieroit little (maybe stupid) idea. If we train a dimensionality reduction model like UMAP on large datasets and use it to compress the embeddings from different embedder to the same...

Create new VectorStore collections whenever the embedder changes

These are placeholders: * https://stackoverflow.com/questions/60290296/word2vec-compare-vectors-from-different-models-with-different-sizes * https://medium.com/@nils_reimers/openai-gpt-3-text-embeddings-really-a-new-state-of-the-art-in-dense-text-embeddings-6571fe3ec9d9 * https://www.philschmid.de/optimize-sentence-transformers

Create new VectorStore collections whenever the embedder changes

@pieroit another placeholder: https://python.langchain.com/en/latest/modules/models/text_embedding/examples/self-hosted.html now the situation is more clear for me

Create new VectorStore collections whenever the embedder changes

I have added qdrant aliases in the last PR. currently tag collections to avoid 2 embedders with the same size but you can think of other ways to use them...

How to do Hybrid search : keyword search + semantic search + reranking ?

Hi @kacperlukawski, maybe I'm off topic but is possible to apply only rerank step? (Qdrant Search + Cross Encoder). Can you recommend any resources where this is discussed? Thanks in...

Add enviroment variable for openai timeout, backoff and max retries

In 1.25 there are these environment variables ``` # Any remote API (OpenAI, Cohere etc.) HAYSTACK_REMOTE_API_BACKOFF_SEC = "HAYSTACK_REMOTE_API_BACKOFF_SEC" HAYSTACK_REMOTE_API_MAX_RETRIES = "HAYSTACK_REMOTE_API_MAX_RETRIES" HAYSTACK_REMOTE_API_TIMEOUT_SEC = "HAYSTACK_REMOTE_API_TIMEOUT_SEC" HAYSTACK_PROMPT_TEMPLATE_ALLOWED_FUNCTIONS = "HAYSTACK_PROMPT_TEMPLATE_ALLOWED_FUNCTIONS" ``` https://github.com/deepset-ai/haystack/blob/v1.25.5/haystack/environment.py I...

Add enviroment variable for openai timeout, backoff and max retries

https://discord.com/invite/FKYDefGP

training, deploying sentence transformer model ( setfit method) in sagemaker pipeline

Hi @miteshkotak , it's hard to tell from this error for me. I used sagemaker library only for deploying from S3 never for train but I think it can be...