emt icon indicating copy to clipboard operation
emt copied to clipboard

verified publisher icon roife
→

Metadata

Emacs macOS Tokenizer, tokenizing CJK words with macOS's built-in NLP tokenizer.

Results 1 emt issues
Sort by recently updated
recently updated
newest added

How to customize the tokenizer default behavior?

1 comment

Excellent work! It seems that NLTokenizer split text but ignored the sign char . example：emt-lib-path，the out put is emt, -lib ,-path but expect is emt ，-，lib，-，path Did there have any...

sincebyte avatar sincebyte

← Metadata

20
Stars
1
Forks
Watchers

Owner

verified publisher icon roife

Metadata

Emacs macOS Tokenizer, tokenizing CJK words with macOS's built-in NLP tokenizer.

Back