legacy-python-cli icon indicating copy to clipboard operation
legacy-python-cli copied to clipboard

UnicodeDecodeError: 'utf8' codec can't decode byte 0xe7 in position 27: invalid continuation byte

Open reklatsmasters opened this issue 7 years ago • 7 comments

Windows 10, Visual Code 1.12.2, WakaTime v1.1.6, wakatime-core 8.0.1

Traceback (most recent call last):
  File "C:\develop\tools\python27\lib\logging\__init__.py", line 859, in emit
    msg = self.format(record)
  File "C:\develop\tools\python27\lib\logging\__init__.py", line 732, in format
    return fmt.format(record)
  File "C:\Users\***\.vscode\extensions\WakaTime.vscode-wakatime-1.1.6\out\wakatime-master\wakatime\logger.py", line 56, in format
    return json.dumps(data)
  File "C:\Users\***\.vscode\extensions\WakaTime.vscode-wakatime-1.1.6\out\wakatime-master\wakatime\packages\simplejson\__init__.py", line 380, in dumps
    return _default_encoder.encode(obj)
  File "C:\Users\***\.vscode\extensions\WakaTime.vscode-wakatime-1.1.6\out\wakatime-master\wakatime\packages\simplejson\encoder.py", line 277, in encode
    chunks = list(chunks)
  File "C:\Users\***\.vscode\extensions\WakaTime.vscode-wakatime-1.1.6\out\wakatime-master\wakatime\packages\simplejson\encoder.py", line 655, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "C:\Users\***\.vscode\extensions\WakaTime.vscode-wakatime-1.1.6\out\wakatime-master\wakatime\packages\simplejson\encoder.py", line 584, in _iterencode_dict
    yield _encoder(value)
  File "C:\Users\***\.vscode\extensions\WakaTime.vscode-wakatime-1.1.6\out\wakatime-master\wakatime\packages\simplejson\encoder.py", line 66, in py_encode_basestring_ascii
    s = s.decode('utf-8')
  File "C:\develop\tools\python27\lib\encodings\utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe7 in position 27: invalid continuation byte
Logged from file logger.py, line 62

reklatsmasters avatar May 20 '17 21:05 reklatsmasters

What file were you editing at that time? Did the file name or contents include any non-ascii characters?

alanhamlett avatar May 21 '17 03:05 alanhamlett

Yes, path to file contains non-ascii characters.

reklatsmasters avatar May 21 '17 07:05 reklatsmasters

Which non-ascii characters does it contain?

Also, what's the value of sys.getdefaultencoding() on your computer?

alanhamlett avatar May 21 '17 16:05 alanhamlett

Also, does the unicode filename test test pass on your machine? To check run these commands:

git clone https://github.com/wakatime/wakatime.git
cd wakatime
virtualenv venv
. venv/bin/activate
pip install -r dev-requirements.txt
nosetests

alanhamlett avatar May 21 '17 16:05 alanhamlett

that was cyrillic characters. sys.getdefaultencoding() returns ascii. The unicode filename test failed.

======================================================================
ERROR: test_nonascii_filename (tests.test_main.MainTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "c:\develop\tools\python27\lib\site-packages\testfixtures\utils.py", line 44, in wrapping
    return wrapped(*args, **kw)
  File "D:\Develop\wakatime\tests\test_main.py", line 498, in test_nonascii_filename
    shutil.copy(entity, os.path.join(tempdir, filename))
  File "c:\develop\tools\python27\lib\shutil.py", line 119, in copy
    copyfile(src, dst)
  File "c:\develop\tools\python27\lib\shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'tests/samples/codefiles/unicode\\a-unicode-file.txt'

reklatsmasters avatar May 21 '17 16:05 reklatsmasters

That test failure was a bug in the tests, can you run git pull inside the wakatime folder then re-run tests with nosetests?

alanhamlett avatar May 22 '17 00:05 alanhamlett

Now the unicode filename test pass, but i have 15 other tests failed with UnicodeDecodeError exception https://pastebin.com/kWdhij0K.

reklatsmasters avatar May 22 '17 10:05 reklatsmasters