GSM8K few-shot sampling from test set?

Open GasolSun36 opened this issue 2 years ago • 1 comments

Excellent work! However, in your paper page 19, Appendix D, you show automatically constructed demonstrations for GSM8K. However, I find that these 8 cases are from test.jsonl but not train.jsonl. Is there a data leak problem?

Mar 08 '23 13:03 GasolSun36

Hi, there would not be a data leak problem because no gold label is used. We only collect questions for automatic rationale generation.

Oct 15 '23 08:10 cooelf