yake icon indicating copy to clipboard operation
yake copied to clipboard

Is Yake suitable for Chinese corpus?

Open Charimanhua opened this issue 1 year ago • 2 comments

Charimanhua avatar Nov 12 '24 02:11 Charimanhua

No, yake is based on segtok, which is fine for indo-european languages. You could fork and integrate with different tokenizer though.

xamgore avatar Jan 11 '25 16:01 xamgore

@Charimanhua check https://github.com/showheroes/yake/pull/1#issue-1767753330

xamgore avatar Jan 13 '25 15:01 xamgore