panphon icon indicating copy to clipboard operation
panphon copied to clipboard

Encoding error on Windows?

Open quadrismegistus opened this issue 7 months ago • 1 comments

Hi there, thanks for this great library. I am using it in prosodic, which I am trying to make compatible with Windows. I am new to Windows development so struggling a bit with some encoding issues in my library and dependencies. I have run into this error:

Python 3.9.0 (default, Nov 15 2020, 08:30:55) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import panphon
>>> ft = panphon.FeatureTable()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\timea\Anaconda3\envs\py39\lib\site-packages\panphon\featuretable.py", line 62, in __init__
    self.segments, self.seg_dict, self.names = self._read_bases(bases_fn, self.weights)
  File "C:\Users\timea\Anaconda3\envs\py39\lib\site-packages\panphon\featuretable.py", line 81, in _read_bases
    header = next(reader)
  File "C:\Users\timea\Anaconda3\envs\py39\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 970: character maps to <undefined>

Is this a matter of specifying the encoding? If so I'm happy to try to make a PR fix.

quadrismegistus avatar Aug 04 '24 11:08 quadrismegistus