giskard
giskard copied to clipboard
[GSK-2824] Difficulty level tests for rags
- Add the possibility to store metadata for each question in the
QATestset
. - Move difficulty as metadata.
- Add possibility to split a test suite into several test based on a metadata :
testset.to_test_suite(slicing_metadata=["difficulty"])
will create a suite with as many tests as difficulty levels in the testset. Each test contains a filtered dataset with only one difficulty level.