PyRIT
PyRIT copied to clipboard
Azure
The Python Risk Identification Tool for generative AI (PyRIT) is an open access automation framework to empower security professionals and machine learning engineers to proactively find risks in their...
Ideally this should apply to test/build pipelines for PRs but integration tests would be nice, too.
#### Is your feature request related to a problem? Please describe. I want to be able to run multi-turn attacks on multiple datasets in parallel that can scale to multiple...
#### Describe the bug In my poetry project I had to set these settings: ``` [[tool.mypy.overrides]] module = ["pyrit.*"] ignore_missing_imports = true ``` If I don't ignore pyrit then I...
It would be great to have an RSS feed that retrieves content from out blog https://azure.github.io/PyRIT/blog/README.html I tried for an hour or two messing with `_config.yaml` but couldn't figure out...
#### Describe the issue linked to the documentation This is a common question. The page should at least describe - all the requirements for a target - multi-turn vs single-turn...
#### Describe the issue linked to the documentation It would be cool if we cross-linked from API reference to user guide examples that use a class, e.g., if the `PromptSendingOrchestrator`...
#### Describe the issue linked to the documentation Lots of parts of the API reference has missing or malformed text. Examples: - https://azure.github.io/PyRIT/api.html  4/5 entries are empty here so...
#### Is your feature request related to a problem? Please describe. We've refactored TreeOfAttacksWithPruningOrchestrator to be a MultiTurnOrchestrator. It would be great if we could have it support image targets...
#### Describe the bug https://github.com/Azure/PyRIT/blob/ecb51c1438aeecb74b45340e4d3993bf0cf688a0/pyrit/datasets/score/refusal.yaml#L81 This line should be removed. The refusal scorer examples need to be consistent with our expectations for it to work. This example 7 has two...
#### Is your feature request related to a problem? Please describe. Paper: https://arxiv.org/pdf/[2401.15817](https://arxiv.org/pdf/2401.15817) We should support this in PyRIT! #### Describe the solution you'd like The most straightforward solution I...