Unable to instantiate HuggingfaceEmbeddings
[x] I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug
Unable to instantiate HuggingfaceEmbeddings, because it doesn't implement asynchronous embed methods of base BaseRagasEmbeddings class.
Ragas version: 0.2.9 Python version: 3.12.8
Code to Reproduce Code is taken from the docs basically.
from ragas.dataset_schema import SingleTurnSample
from ragas.embeddings import HuggingfaceEmbeddings
from ragas.metrics import SemanticSimilarity
sample = SingleTurnSample(
response="The Eiffel Tower is located in Paris.",
reference="The Eiffel Tower is located in Paris. I has a height of 1000ft."
)
ragas_embeddings = HuggingfaceEmbeddings(model_name="BAAI/bge-reranker-v2-m3")
scorer = SemanticSimilarity(embeddings=ragas_embeddings, is_cross_encoder=True)
await scorer.single_turn_ascore(sample)
Error trace
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
File /home/magnesium/sadaivault/prrag/kilt_eval.py:[1](https://file+.vscode-resource.vscode-cdn.net/home/magnesium/sadaivault/prrag/kilt_eval.py:1)
----> 1 ragas_embeddings = HuggingfaceEmbeddings(model_name="BAAI/bge-reranker-v2-m3")
TypeError: Can't instantiate abstract class HuggingfaceEmbeddings without an implementation for abstract methods 'aembed_documents', 'aembed_query'
Expected behavior I expected the object to be instantiated and functional.
hey @sadaisystems could you maybe check out fastembed https://github.com/qdrant/fastembed it had async support last time I checked, which was a while back, could you check it out?
let me know if you need help. I'll try it out in my end and share the code snippet too
Try this
from ragas.dataset_schema import SingleTurnSample
from ragas.metrics import SemanticSimilarity
from langchain_huggingface import HuggingFaceEmbeddings
sample = SingleTurnSample(
response="The Eiffel Tower is located in Paris.",
reference="The Eiffel Tower is located in Paris. I has a height of 1000ft."
)
langchain_hf_embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
ragas_embeddings = LangchainEmbeddingsWrapper(langchain_hf_embeddings)
scorer = SemanticSimilarity(embeddings=ragas_embeddings, is_cross_encoder=True)
await scorer.single_turn_ascore(sample)