Azure
FEAT Add JailbreakV_28K dataset
Is your feature request related to a problem? Please describe.
Add this dataset into PyRIT - it is not currently apart of PyRIT yet. https://huggingface.co/datasets/JailbreakV-28K/JailBreakV-28k/viewer/JailBreakV_28K/mini_JailBreakV_28K?views%5B%5D=jailbreakv_28k_mini_jailbreakv_28k&row=4
Describe the solution you'd like
The HuggingFace data set: https://huggingface.co/datasets/JailbreakV-28K/JailBreakV-28k/viewer/JailBreakV_28K/mini_JailBreakV_28K?views%5B%5D=jailbreakv_28k_mini_jailbreakv_28k&row=4 The associated paper: https://arxiv.org/abs/2404.03027
Additional context
Similar to previous dataset contributions this should live in pyrit.datasets as a "fetch" function. Also, the harm_categories property should be set on each prompt. This huggingface dataset mentions each harm category under the "policy" column. There are examples of how PyRIT interacts with other datasets here: https://github.com/search?q=repo%3AAzure%2FPyRIT%20%23%20The%20dataset%20sources%20can%20be%20found%20at%3A&type=code
[[Content Warning: Prompts are aimed at provoking the model, and may contain offensive content.]] Additional Disclaimer: Given the content of these prompts, keep in mind that you may want to check with your relevant legal department before trying them against LLMs.
