FunASR icon indicating copy to clipboard operation
FunASR copied to clipboard

求教tokens.json的生成和问题?

Open xiulianzw opened this issue 1 year ago • 0 comments
trafficstars

Notice: In order to resolve issues more efficiently, please raise issue following the template. (注意:为了更加高效率解决您遇到的问题,请按照模板提问,补充细节)

❓ Questions and Help

看了一下modelscope的越南语UniASR的预训练模型的token.json文件,里面的token有大量包含了 "<OOV>", "<@@", ">", "-@@", "'", "'@@", "a", "a@@", token文件里面的@符号有什么特殊的含义吗?我用aishell/paraformer/utils/text2token.py生成的token文件里面就没有@符号相关的

xiulianzw avatar Apr 09 '24 10:04 xiulianzw