Parsing the output of a model

[jylong91]

The output of the model largely depends on the prompt. When performing text classification tasks such as sst2, which requires the model to give negative or positive responses, the model may provide various forms of answers, and the form of answers from different models may vary. Is there a good way to parse these results?

[Immortalise]

Hi, our current approach involves directing the model to specifically output the desired labels ('positive' or 'negative'), which are then enclosed within a unique pattern (e.g., '<<<>>>'). This format allows for easy extraction using regular expressions.

[Rizwanali324]

im getting issues regarding qqp dataset evaluation for diff attacks. it gives me dataset_name attribute error when I run attacks

[Immortalise]

Hi, I have tested qqp dataset with Flan-T5 model using StressTest attack, it works for me. Could you please test it and paste the detailed error messages here?

[github-actions[bot]]

Stale issue message