ragas
ragas copied to clipboard
is_async missing in context_relevancy in ragas 0.1.8
In ragas 0.1.8 in _context_relevancy.py
async def _ascore(self, row: t.Dict, callbacks: Callbacks, is_async: bool) -> float:
assert self.llm is not None, "LLM is not initialized"
if self.show_deprecation_warning:
logger.warning(
"The 'context_relevancy' metric is going to be deprecated soon! Please use the 'context_precision' metric instead. It is a drop-in replacement just a simple search and replace should work." # noqa
)
question, contexts = row["question"], row["contexts"]
result = await self.llm.generate(
self.context_relevancy_prompt.format(
question=question, context="\n".join(contexts)
),
callbacks=callbacks
)
return self._compute_score(result.generations[0][0].text, row)
I think self.llm.generate is missing is_async=is_async. I don't know if this is a bug or not, but found it weird it was the only one who lacked the argument and in my setup is failing only this metric.
Thank you in advance!