SudachiPy icon indicating copy to clipboard operation
SudachiPy copied to clipboard

UnicodeDecodeError

Open KHiyowa opened this issue 6 years ago • 0 comments

SudachiPy Command Line Ver on Cygwin Terminal.

Type: 貴社の記者が汽車で帰社する [Enter]

貴社の記者が汽車で帰社する
貴社    名詞,普通名詞,一般,*,*,*        貴社
の      助詞,格助詞,*,*,*,*     の
記者    名詞,普通名詞,一般,*,*,*        記者
が      助詞,格助詞,*,*,*,*     が
汽車    名詞,普通名詞,一般,*,*,*        汽車
で      助詞,格助詞,*,*,*,*     で
帰社    名詞,普通名詞,サ変可能,*,*,*    帰社
する    動詞,非自立可能,*,*,サ行変格,終止形-一般        為る
EOS

Type: 貴社の記者が汽車で帰社する [Back Space] [Enter]

貴社の記者が汽車で帰社す
  File "~/bin/sudachipy", line 11, in <module>
    sys.exit(main())
  File "~/lib/python3.7/site-packages/sudachipy/command_line.py", line 235, in main
    args.handler(args, args.print_usage)
  File "~/lib/python3.7/site-packages/sudachipy/command_line.py", line 173, in _command_tokenize
    run(tokenizer_obj, mode, input_, print_all, stdout_logger, enable_dump)
  File "/lib/python3.7/site-packages/sudachipy/command_line.py", line 61, in run
    for line in input_:
  File "~/lib/python3.7/fileinput.py", line 252, in __next__
    line = self._readline()
  File "~/lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 36-37: invalid continuation byte

Sudachipy: 0.3.13 / Dict: 20190718_core

KHiyowa avatar Sep 04 '19 02:09 KHiyowa