ragas icon indicating copy to clipboard operation
ragas copied to clipboard

Unable to instantiate HuggingfaceEmbeddings

Open sadaisystems opened this issue 11 months ago • 2 comments

[x] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug Unable to instantiate HuggingfaceEmbeddings, because it doesn't implement asynchronous embed methods of base BaseRagasEmbeddings class.

Ragas version: 0.2.9 Python version: 3.12.8

Code to Reproduce Code is taken from the docs basically.

from ragas.dataset_schema import SingleTurnSample
from ragas.embeddings import HuggingfaceEmbeddings
from ragas.metrics import SemanticSimilarity

sample = SingleTurnSample(
    response="The Eiffel Tower is located in Paris.",
    reference="The Eiffel Tower is located in Paris. I has a height of 1000ft."
)
ragas_embeddings = HuggingfaceEmbeddings(model_name="BAAI/bge-reranker-v2-m3")
scorer = SemanticSimilarity(embeddings=ragas_embeddings, is_cross_encoder=True)

await scorer.single_turn_ascore(sample)

Error trace

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File /home/magnesium/sadaivault/prrag/kilt_eval.py:[1](https://file+.vscode-resource.vscode-cdn.net/home/magnesium/sadaivault/prrag/kilt_eval.py:1)
----> 1 ragas_embeddings = HuggingfaceEmbeddings(model_name="BAAI/bge-reranker-v2-m3")

TypeError: Can't instantiate abstract class HuggingfaceEmbeddings without an implementation for abstract methods 'aembed_documents', 'aembed_query'

Expected behavior I expected the object to be instantiated and functional.

sadaisystems avatar Jan 04 '25 13:01 sadaisystems

hey @sadaisystems could you maybe check out fastembed https://github.com/qdrant/fastembed it had async support last time I checked, which was a while back, could you check it out?

let me know if you need help. I'll try it out in my end and share the code snippet too

jjmachan avatar Jan 07 '25 10:01 jjmachan

Try this

from ragas.dataset_schema import SingleTurnSample
from ragas.metrics import SemanticSimilarity
from langchain_huggingface import HuggingFaceEmbeddings

sample = SingleTurnSample(
    response="The Eiffel Tower is located in Paris.",
    reference="The Eiffel Tower is located in Paris. I has a height of 1000ft."
)
langchain_hf_embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
ragas_embeddings = LangchainEmbeddingsWrapper(langchain_hf_embeddings)
scorer = SemanticSimilarity(embeddings=ragas_embeddings, is_cross_encoder=True)

await scorer.single_turn_ascore(sample)

ZenBel avatar Mar 11 '25 16:03 ZenBel