FEAT: CBT-Bench Dataset

[jbolor21]

Adding this dataset of psychotherapy benchmark questions: https://huggingface.co/datasets/Psychotherapy-LLM/CBT-Bench

Describe the solution you'd like:

This dataset can be fetched and used in PyRIT!

[sravan0446]

I'd like to add support for the CBT-Bench dataset by implementing a new dataset loader (CBTBenchDataset) in PyRIT. This dataset will be accessible through the standard dataset interface, enabling red teamers to evaluate generative models using psychotherapy-related prompts.

The solution includes:

A new dataset connector in pyrit/datasets/cbt_bench.py

Adding entry in init.py

A test file in tests/ to verify correct loading and structure if needed

This will allow users to easily incorporate psychotherapy benchmark questions into their red teaming pipelines with minimal setup.

Let me know if i got anything wrong, you can brief me.

[romanlutz]

Sounds great! Just make sure the unit test doesn't actually make a network call. We do the full test in the integration tests (where you can follow the established pattern for datasets).

That means mocking the corresponding function that makes the call. If you aren't sure, check other dataset fetching unit tests or just open the PR and I'll comment there 🙂

Thank you for contributing!

[sravan0446]

Raised a PR: #888 to integrate the CBT-Bench dataset into PyRIT.

I haven't added unit test support for this, as other Hugging Face datasets currently don't have associated unit tests either. However, I'm happy to add them if needed — please let me know.