MRM-pytorch icon indicating copy to clipboard operation
MRM-pytorch copied to clipboard

Published 20 hours ago • verified publisher icon RL4M

How to generate my own wordpiece.json

Open ToniChopp opened this issue 1 year ago • 0 comments

Hello, I have found that there are many semantic unrelated tokens in mimic_wordpiece.json. So I want to generate my own wordpiece file. How could I achieve it?

ToniChopp avatar Apr 25 '23 07:04 ToniChopp