haystack-core-integrations
haystack-core-integrations copied to clipboard
SerializationError when using RagasEvaluator with `ragas_metrics`
Describe the bug
When using the RagasEvaluator component with the ragas_metrics parameter, serializing the pipeline (e.g., using pipeline.dumps()) raises a SerializationError. This occurs because the class does not assign ragas_metrics to a public instance variable, nor implements a custom to_dict() method.
Error Trace
haystack.core.errors.SerializationError: Cannot determine the value of the init parameter 'ragas_metrics' for the class RagasEvaluator. You can fix this error by assigning 'self.ragas_metrics = ragas_metrics' or adding a custom serialization method 'to_dict' to the class.
To Reproduce
from haystack import Pipeline
import os
os.environ["AZURE_OPENAI_API_KEY"] = "your-api-key"
os.environ["AZURE_OPENAI_ENDPOINT"] = "your-endpoint"
from ragas.llms import HaystackLLMWrapper
from ragas.metrics import AnswerRelevancy, ContextPrecision, Faithfulness
from haystack_integrations.components.evaluators.ragas import RagasEvaluator
from haystack.components.generators.azure import AzureOpenAIGenerator
llm = AzureOpenAIGenerator()
evaluator_llm = HaystackLLMWrapper(llm)
ragas_evaluator = RagasEvaluator(
ragas_metrics=[AnswerRelevancy(), ContextPrecision(), Faithfulness()],
evaluator_llm=evaluator_llm,
)
pipeline = Pipeline()
pipeline.add_component("ragas_evaluator", ragas_evaluator)
print(pipeline.dumps())
Expected behavior
The pipeline should serialize successfully, including the RagasEvaluator component.
Proposed Fix
in the RagasEvaluator component, for better naming consistency, the parameter itself could be renamed from ragas_metrics to metrics, matching the internal attribute:
def __init__(
self,
metrics: List[Metric],
evaluator_llm: Optional[BaseRagasLLM] = None,
evaluator_embedding: Optional[BaseRagasEmbeddings] = None,
):
self.metrics = metrics