slurp icon indicating copy to clipboard operation
slurp copied to clipboard

Intents sometimes missing the scenario

Open PabloPeso opened this issue 1 year ago • 1 comments

I was looking at the gold data in SLURP jsonl format placed https://github.com/pswietojanski/slurp/blob/master/dataset/slurp/train.jsonl and I noticed there are a few intents with very low examples, as for example podcasts or wemo_off

As far as I understand, the intent is created with scenario_action, but in these two cases the scenario was ignored when created the intent. Is this a bug or it was made on purpose?

I can see there are quite a lot more examples of play_podcasts and iot_wemo_off, which would the intent for the previous cases.

This happens also in the https://github.com/pswietojanski/slurp/blob/master/dataset/slurp/test.jsonl which will be affecting the performance results I think, is this right?

Thanks for you help.

PabloPeso avatar Mar 17 '23 13:03 PabloPeso