embedding factory

[amin-kh96]

I checked the evaluate function and deep inside there is a function called 'embedding_factory'. by default it use openai I put the whole function here.

def embedding_factory( model: str = "text-embedding-ada-002", run_config: t.Optional[RunConfig] = None ) -> BaseRagasEmbeddings: openai_embeddings = OpenAIEmbeddings(model=model) if run_config is not None: openai_embeddings.request_timeout = run_config.timeout else: run_config = RunConfig() return LangchainEmbeddingsWrapper(openai_embeddings, run_config=run_config)

and I did this also:

Assuming custom_embeddings is an instance of your CustomHuggingFaceRagasEmbeddings

evaluation_report = evaluate(ragas_data, metrics=metrics, embeddings=custom_embeddings)

but it says the same error of API again, I want to substitute the output of a function called embedded factory with my computed embeddings. because I already have the computed embeddings both for the answer and the question and also the context in a Ragas data frame. I have every number (float) needed for the computation. but ragas evaluate function does the computation again.

[dosubot[bot]]

It seems like you're trying to use your precomputed embeddings with the evaluate function in Ragas, but you're encountering an issue where the function still attempts to use the default OpenAI embeddings. Here are some similar issues and solutions that might help:

1. Issue evaluate function: You can modify the behavior of the evaluate function to use your custom embeddings by passing them directly to the function. This bypasses the embedding_factory function. Ensure your custom embeddings class is correctly implemented and instantiated, then pass it to the evaluate function using the embeddings parameter:

   # Assuming `custom_embeddings` is an instance of your CustomHuggingFaceRagasEmbeddings
   evaluation_report = evaluate(ragas_data, metrics=metrics, embeddings=custom_embeddings)
   

   By providing the embeddings parameter, the evaluate function will use your custom embeddings and will not call the embedding_factory function, thus avoiding the default OpenAI embeddings ^[1].

2. Issue embeddings: To use your dataset of embeddings, implement the embed_documents and embed_query methods in your CustomEmbeddings class. Store your embeddings in a dictionary with keys like 'questions', 'answers', and 'contexts', and ensure these methods return the appropriate embeddings. Use your CustomEmbeddings class with the evaluate function, adjusting the logic to fit your data structure ^[2].

Make sure your custom embeddings are correctly integrated and that the evaluate function is configured to use them. If the error persists, double-check the implementation of your custom embeddings class and the way it's being passed to the evaluate function.

To continue talking to Dosu, mention @dosu.

[amin-kh96]

no no, you don't get it. I want to substitute the output of a function called embedded factory with my computed embeddings. because I already have the computed embeddings both for the answer and the question and also the context in a Ragas data frame. I have every number (float) needed for the computation. but ragas evaluate function does the computation again.

[dosubot[bot]]

To substitute the output of the embedding_factory function with your precomputed embeddings in the evaluate function, you need to ensure that your precomputed embeddings are passed directly to the evaluate function using the embeddings parameter. This will prevent the evaluate function from calling the embedding_factory and using the default OpenAI embeddings.

Here's how you can do it:

1. Prepare Your Precomputed Embeddings: Ensure your precomputed embeddings are structured in a way that the evaluate function can use. This typically involves implementing a class that mimics the expected interface, such as having methods like embed_documents and embed_query.

2. Pass Precomputed Embeddings to Evaluate: When calling the evaluate function, pass your precomputed embeddings directly:

   # Assuming `precomputed_embeddings` is an instance of your custom embeddings class
   evaluation_report = evaluate(ragas_data, metrics=metrics, embeddings=precomputed_embeddings)
   

By doing this, the evaluate function will use your provided embeddings and will not invoke the embedding_factory function, thus avoiding recomputation of embeddings. Make sure your custom embeddings class is correctly implemented to fit the expected interface.

To continue talking to Dosu, mention @dosu.

[anistark]

Check out llm_factory with from ragas.embeddings.base import BaseRagasEmbedding

[github-actions[bot]]

It seems the issue was answered, closing this now.