Add Red-Teaming LLMs

[dapurv5]

https://github.com/dapurv5/awesome-red-teaming-llms

Red-teaming techniques and defenses for Large Language Models (LLMs).

By submitting this pull request I confirm I've read and complied with the below requirements 🖖

Requirements for your pull request

• [x] Don't open a Draft / WIP pull request while you work on the guidelines. A pull request should be 100% ready and should adhere to all the guidelines when you open it.
• [x] You have to review at least 2 other open pull requests. I have reviewed the following PRs: #3234 and #3232.
• [x] You have read and understood the instructions for creating a list.
• [x] This pull request has a title in the format Add Name of List.
• [x] Your entry here should include a short description of the project/theme of the list.
• [x] Your entry should be added at the bottom of the appropriate category.
• [x] The title of your entry should be title-cased and the URL to your list should end in #readme.
• [x] No blockchain-related lists.
• [x] The suggested Awesome list complies with the below requirements.

Requirements for your Awesome list

• [x] Has been around for at least 30 days.
• [x] Run awesome-lint on your list and fix the reported issues.
• [x] The default branch should be named main, not master.
• [x] Includes a succinct description of the project/theme at the top of the readme.
• [x] It's the result of hard work and the best I could possibly produce.
• [x] The repo name of your list should be in lowercase slug format: awesome-name-of-list.
• [x] The heading title of your list should be in title case format: # Awesome Name of List.
• [x] Non-generated Markdown file in a GitHub repo.
• [x] The repo should have awesome-list & awesome as GitHub topics.
• [x] Not a duplicate.
• [x] Only has awesome items. Awesome lists are curations of the best, not everything.
• [x] Does not contain items that are unmaintained, has archived repo, deprecated, or missing docs.
• [x] Includes a project logo/illustration whenever possible.
• [x] Entries have a description, unless the title is descriptive enough by itself.
• [x] Includes the Awesome badge.
• [x] Has a Table of Contents section.
• [x] Has an appropriate license.
• [x] Has contribution guidelines.
• [x] All non-important but necessary content is grouped in a Footnotes section at the bottom of the readme.
• [x] Has consistent formatting and proper spelling/grammar.
• [x] Does not use hard-wrapping.
• [x] Does not include a CI (e.g. GitHub Actions) badge.
• [x] Does not include an Inspired by awesome-foo or Inspired by the Awesome project kinda link at the top of the readme.

[dapurv5]

unicorn

[sindresorhus]

Thanks for making an Awesome list! 🙌

It looks like you didn't read the guidelines closely enough. I noticed multiple things that are not followed. Try going through the list point for point to ensure you follow it. I spent a lot of time creating the guidelines so I wouldn't have to comment on common mistakes, and rather spend my time improving Awesome.

[dapurv5]

Thank you so much for your comment as I was not able to figure out the issue myself. I had a the list-based version earlier which did not do justice to presenting the information, so I decided to change it to a table. Any easy resolution for this?

[Axorax]

You should add a short description at the top of the README so people can read and understand it quickly

[dapurv5]

• Added a one line summary at the top
• Expanded the table of contents to reflect all the headings in the taxonomy.

[thisisisa]

Hi @dapurv5, thank you for updating the PR! Here are a couple more things I noticed on your list:

• I think the Introduction and Attack Surface sections are too long. I would recommend removing them and instead adding a brief definition of the theme at the top of the list, and this description would be better presented as a quote.
• The table of contents should then start at Attacks, where the list begins, with a maximum depth of one. You can retain titles with greater depth within the list itself, but they should not appear in the table of contents.

Contents

• Attacks
  • Jailbreak Attack
  • Direct Attack
  • Infusion Attack
  • Inference Attack
  • Training Time Attack
• Defenses

[dapurv5]

Thanks for your suggestions and helpful pointers @idematos . I ran the linter offline and it passes now.

[sindresorhus]

The list does not follow the expected format of Awesome lists.

The list should not be nested within sub-pages.

[colbytimm]

Can you add attacks and defence tables in the main readme instead of having them as separate markdown files. Also wonder if you should link the paper as a reference in the readme vs having it in the description.