sengiri icon indicating copy to clipboard operation
sengiri copied to clipboard

Can't tokenize

Open Subrata15 opened this issue 3 years ago • 0 comments

Hi there! I can't tokenize the sentence, and raised error "ValueError: too many values to unpack (expected 2)" Im just try this code :

print(sengiri.tokenize('遅刻したけど楽しかったし嬉しかった。すごく充実した!'))

and surface this error

Subrata15 avatar Mar 30 '21 02:03 Subrata15