giskard icon indicating copy to clipboard operation
giskard copied to clipboard

[GSK-2824] Difficulty level tests for rags

Open pierlj opened this issue 4 months ago • 2 comments

  • Add the possibility to store metadata for each question in the QATestset.
  • Move difficulty as metadata.
  • Add possibility to split a test suite into several test based on a metadata : testset.to_test_suite(slicing_metadata=["difficulty"]) will create a suite with as many tests as difficulty levels in the testset. Each test contains a filtered dataset with only one difficulty level.

pierlj avatar Feb 16 '24 15:02 pierlj