YangjiaJoJo

@quhonglin I think you should get a unit LM, which means that the model use char as unit to build

@Abdullah955 I follows the right steps. And the lexicon format is just as above. But when it goes to " self.lm = KenLM(cfg.lmpath, self.word_dict)" . Segmentation fault (core dumped) occurs....

The model is data2vec base model.