pyment
pyment copied to clipboard
dadadel
encoding error on windows
I'm using pyment on windows with python3
All my python files are utf-8 encoded, and when I try to use it on my codes with CJK, I got UnicodeDecodeError: 'gbk' codec can't decode byte 0xa5 in position 408: illegal multibyte sequence
https://github.com/dadadel/pyment/blob/3d1bdf87d083ff56230bd0bf7c5252e20552b7b6/pyment/pyment.py#L87-L97
Maybe open(self.input_file, encoding='utf-8')
Hi @Trim21 Thanks for reporting. Did you try the fix you suggested? And if so, did it work? Have you some code sample? When I have utf-8 in a script, I have no error. But I am on Linux.
It's a little bit complicated. It works with python3 on windows if users are using utf-8 as encoding. (I'm doing this) But some users may use gbk as encoding, which is the default encoding when calling open.
I'm considering writing a package to detect python source file encoding for these problems.
Oh. I was wondering why ci on windows passed all tests. Then I found that none of them include any CJK characters.
Hi @Trim21 Thanks for reporting. Did you try the fix you suggested? And if so, did it work? Have you some code sample? When I have utf-8 in a script, I have no error. But I am on Linux.
this patch works for me
