ragas icon indicating copy to clipboard operation
ragas copied to clipboard

fix: update required columns in RefusalRate (docs)

Open DmitrySirakov opened this issue 8 months ago • 3 comments

Overview

This PR fixes a bug in the RefusalRate docs metric implementation. The ValidationError occurred because the required fields for a SingleTurnSample were set to {"response", "reference"}, while the metric uses the "user_input" field for evaluation. This mismatch causes a validation error when creating a RefusalInput instance.

Changes Made

  • Updated the _required_columns for MetricType.SINGLE_TURN in RefusalRate:
    • From: {"response", "reference"}
    • To: {"response", "user_input"}
  • This change ensures that the "user_input" field is present during validation, preventing any ValidationError.

Issue Reference

This addresses the issue described in issue #1999.

DmitrySirakov avatar Apr 13 '25 20:04 DmitrySirakov