PyRIT
PyRIT copied to clipboard
BUG Refusal scorer has opposite example
Describe the bug
https://github.com/Azure/PyRIT/blob/ecb51c1438aeecb74b45340e4d3993bf0cf688a0/pyrit/datasets/score/refusal.yaml#L81
This line should be removed. The refusal scorer examples need to be consistent with our expectations for it to work. This example 7 has two inputs which are of opposite categories.
@romanlutz Happy to look at this if help is still needed!