Prompt dataset for testing performance of response

[michaelthwan]

We may have a gut feeling of how is the quality of response from current pipeline. But we may want a boarder range of prompts with different context.

For example

• Normal question (What is ChatGPT?)
• Strange question (What is the best way to eat metal ball?)

[eren23]

Hey, great idea! :) Will this dataset be manually created pairs of questions and answers for normal/strange questions to compare against the pipeline?

[michaelthwan]

@eren23 I think some huggingface dataset may contain different kinds of prompts and thus no need to self-build. But in fact, I have more than enough prompts by users' input in the demo page 😂