slue-toolkit icon indicating copy to clipboard operation
slue-toolkit copied to clipboard
verified publisher icon asappresearch

Dataset Indexing Issue

Open ming024 opened this issue 2 years ago • 0 comments

In the data collator of AddLabelDataset

https://github.com/asappresearch/slue-toolkit/blob/ea0939b0c9da80f19dda35af26e117834fb20ec6/slue_toolkit/fairseq_addon/data/add_label_dataset.py#L28-L33

If self.dataset.collater sort the collated data, target and collated will become mismatched.

ming024 avatar May 18 '23 05:05 ming024