fairseq-detect-hallucination icon indicating copy to clipboard operation
fairseq-detect-hallucination copied to clipboard

Published 20 hours ago • verified publisher icon violet-zct

All the tokens are marked as hallucinations

Open sourabhXIII opened this issue 2 years ago • 0 comments

Hello @violet-zct Thanks for the repo.

I am trying to understand the working of convert_spm_labels_to_raw_labels in the utils prediction scripts. None of the BPE tokens will start with \u2581. Thus in effect concluding that all the tokens are hallucinations.

I must be making a mistake here. Can you please help? Thank you for your time.

sourabhXIII avatar Feb 07 '23 14:02 sourabhXIII