Tenzin

Results 5 issues of Tenzin

I am getting following error. --------------------------------------------------------------------------- ``` OverflowError Traceback (most recent call last) in () 1 boxes = remove_duplicates(boxes, test_image) ----> 2 img = draw_boxes(boxes, test_image) 3 plt.figure(figsize=[10, 10]) 4...

https://github.com/OpenPecha/Botok/blob/master/botok/resources/bo_uni_table.csv

Reproduce script ```python tokens = wt.tokenize("རིན་ཆེན་མིའི") print(tokens) ``` output: ``` [text: "རིན་ཆེན་" text_cleaned: "རིན་ཆེན་" text_unaffixed: "རིན་ཆེན་" syls: ["རིན", "ཆེན"] pos: OTHER lemma: རིན་ཆེན་ senses: | pos: OTHER, freq: 22841, affixed:...

System: - botok: v0.8.8 Reproduce ```python tokens = wt.tokenize("༄༅། །བློ་སྦྱོང་དོན་?") print(tokens[0]) ``` Output ``` text: "༄༅། །" char_types: |NORMAL_PUNCT|NORMAL_PUNCT|NORMAL_PUNCT|TRANSPARENT|NORMAL_PUNCT| chunk_type: PUNCT start: 0 len: 5 ``` Expected output: ``` text:...

```python WT = WordTokenizer() tokens = WT.tokenize(in_str) token = tokens[0] token.get('lemma') ```

enhancement