Convert-PolyAI-Torch icon indicating copy to clipboard operation
Convert-PolyAI-Torch copied to clipboard

Published 20 hours ago • verified publisher icon jordiclive

BPE from scratch

Open jordiclive opened this issue 5 years ago • 0 comments

implement BPE from scratch with unk tokens hashed (although may achieve worse results on downstream tasks) as # perhaps not as general as bpemb's 25000.model

jordiclive avatar Aug 30 '20 13:08 jordiclive