Create test cases (possibly even synthetic data for training) from ChatGPT failure examples (see repo)

[giuven95]

This repo can be used as guidance: https://github.com/giuven95/chaptgt-failures

[yk]

Thank you. I've put this in backlog for now until we have a testing & data pipeline setup to support something like this.

[smytjf11]

I'm wondering if there is value in using it for the safety team. The baseline will be trivial, it's edge cases like this that will be hard to nail down.

[huu4ontocord]

Hi @giuven95 since you are interested in this issue, we are also working to collect chatgpt prompts people have created and is available on the open web. Interested in helping with this?

[Forbu]

Do you think it will be usefull to gather data of OA failures too ? (scraping discord "bad-message-ids" ?)

[andreaskoepf]

Closing old data issue.