AssertionError: Adapted output keys set(output.keys())=set() do not match with the original output keys: output_keys[i]={'statements'}

[Ailce8862]

[ ] I have checked the documentation and related resources and couldn't resolve my bug.

Describe the bug AssertionError: Adapted output keys set(output.keys())=set() do not match with the original output keys: output_keys[i]={'statements'}

Ragas version:0.1.5 Python version:3.10.13

Code to Reproduce from ragas import adapt from ragas.metrics import ( faithfulness, answer_relevancy, context_recall, ) from langchain_community.embeddings import QianfanEmbeddingsEndpoint from langchain_community.chat_models import QianfanChatEndpoint

embed_llm = QianfanEmbeddingsEndpoint(
    qianfan_ak='XXX',
    qianfan_sk='XXX'
)
chat_llm = QianfanChatEndpoint(streaming=True)

adapt(metrics=[faithfulness, answer_relevancy,context_recall], language="Chinese", llm=chat_llm)
print(faithfulness.long_form_answer_prompt.to_string())

Error trace Traceback (most recent call last): File "/Users/liangpan/工作/新点/coding/RAG_langchain/step3/RAGAS_eval_separation.py", line 133, in adapt(metrics=[faithfulness, answer_relevancy,context_recall], language="Chinese", llm=chat_llm) File "/Users/liangpan/miniforge3/envs/langchain/lib/python3.10/site-packages/ragas/adaptation.py", line 36, in adapt metric.adapt(language, cache_dir=cache_dir) File "/Users/liangpan/miniforge3/envs/langchain/lib/python3.10/site-packages/ragas/metrics/_faithfulness.py", line 203, in adapt self.long_form_answer_prompt = self.long_form_answer_prompt.adapt( File "/Users/liangpan/miniforge3/envs/langchain/lib/python3.10/site-packages/ragas/llms/prompt.py", line 231, in adapt set(output.keys()) == output_keys[i] AssertionError: Adapted output keys set(output.keys())=set() do not match with the original output keys: output_keys[i]={'statements'}

Process finished with exit code 1

[LukaszDejneka]

Same issue for trying to adapt to Polish language.

[jjmachan]

tagging #890 as a meta issue to cleanup everything

[jjmachan]

This has been fixed with v0.2 - I know finally 😅 🎉

do checkout the docs here: https://docs.ragas.io/en/stable/howtos/customizations/metrics/_metrics_language_adaptation/ reference here: https://docs.ragas.io/en/stable/references/prompt/#ragas.prompt.PromptMixin

and if you're migrating from v0.1 check out the migration docs here: https://docs.ragas.io/en/stable/howtos/migrations/migrate_from_v01_to_v02

could you check it out and verify if not feel free to comment here and I'll help you out - really sorry again that it tool this while