FlagEmbedding icon indicating copy to clipboard operation
FlagEmbedding copied to clipboard

Published 20 hours ago • verified publisher icon FlagOpen

tokenizer的相关问题

Open iamreallyi9 opened this issue 10 months ago • 4 comments

你好，请教一下我想用自己的tokenizer【chatglm的tokenizer】来训练bge模型应该怎么做呢？我现在的想法是把【chatglm的tokenizer】改成类似bert-tokenizer，这样可行吗？

iamreallyi9 avatar Apr 17 '24 10:04 iamreallyi9