An 'answer_relevancy' error occurred.

[TakutoIyanagi-littletree]

• [x] I checked the documentation and related resources and couldn't find an answer to my question.

Your Question The following error occurred NotImplementedError: adapt() is not implemented for answer_relevancy metric

Code Examples

from ragas.llms import llm_factory
from ragas import evaluate
from datasets import Dataset
from ragas.metrics._answer_relevance import AnswerRelevancy
from ragas.metrics._answer_relevance import ResponseRelevancy

# Ragasの評価用LLMの設定
LLM_NAME = "gpt-4o-mini"
ragas_llm = llm_factory(model=LLM_NAME)

# Adapt the ResponseRelevancy metric to Japanese
answer_relevancy = ResponseRelevancy()
answer_relevancy.adapt(language="japanese")

dataset = Dataset.from_dict(
    {
        "question": questions,
        "answer": answers,
        "contexts": contexts,
        "ground_truth": ground_truths,
    }
)

score = evaluate(
    dataset,
    llm=ragas_llm,
    metrics=[
        answer_relevancy,
    ],
)
print(score)
score.to_pandas()

[dosubot[bot]]

I couldn't find specific information about the implementation status of the adapt method in the AnswerRelevancy class within the Ragas framework. However, the error message you're encountering, NotImplementedError: adapt() is not implemented for answer_relevancy metric, suggests that the adapt method is not yet implemented for the AnswerRelevancy metric.

You might want to check if there are any updates or recent changes in the repository that address this issue. Additionally, you could consider using other metrics that support the adapt method or explore alternative approaches to achieve your goal.

To continue talking to Dosu, mention @dosu.

[anistark]

Try with metrics collections from ragas.metrics.collections import AnswerRelevancy