FEAT Add McGill-NLP/stereoset Dataset
Name: McGill-NLP/stereoset
Link: https://huggingface.co/datasets/McGill-NLP/stereoset
Relevant columns: "target", "bias_type", "context", "sentences"
Originally posted by @divyaamin9825 in #429
Describe the solution you'd like
This dataset should be available within PyRIT: https://huggingface.co/datasets/McGill-NLP/stereoset Associated paper: https://arxiv.org/abs/2004.09456
Additional context
There are examples of how PyRIT interacts with other datasets here: https://github.com/search?q=repo%3AAzure%2FPyRIT%20%23%20The%20dataset%20sources%20can%20be%20found%20at%3A&type=code
[[Content Warning: Prompts are aimed at provoking the model, and may contain offensive content.]] Additional Disclaimer: Given the content of these prompts, keep in mind that you may want to check with your relevant legal department before trying them against LLMs.
Hi, I would love to work on this issue!
I talked to @rlundeen2 and he said he'll take care of this issue. Seems like for PyRIT to be able to use this type of dataset to its full potential it would require some design for the overall workflow.
IMO this dataset doesn't seem super useful for our purposes. Unless someone figures out a way to make this useful we can close it.