ion-python
ion-python copied to clipboard
Reading a literal (non-escaped) emoji doesn't work when reading from a file with mode 'rb'
For a file containing the following symbol:
'🏳0🌈️'
the following behavior is observed:
>>>
>>> with open('<filename>', 'r') as f:
... print(ion.load(f, single_value=False))
...
[IonPySymbol(text='🏳0🌈️', sid=None, location=None)] # correct
>>> with open('<filename>', 'rb') as f:
... print(ion.load(f, single_value=False))
...
[IonPySymbol(text='ð\x9f\x8f³0ð\x9f\x8c\x88ï¸\x8f', sid=None, location=None)] # incorrect
Note that ion-python tests appear to be cheating: https://github.com/amzn/ion-python/blob/master/tests/test_vectors.py#L87
I believe the same symbol should be constructed whether the file is opened with mode 'r' or 'rb'.