grammars-v4
grammars-v4 copied to clipboard
UnicodeDecodeError with Python3 runtime
I have a repository with a Python 3 parser. And there are a couple of problematic files that Java target parses without error, but the Python target throws an error.
problematic files: fractions.py ftplib.py functools.py getopt.py heapq.py random.py runpy.py shlex.py statistics.py
to reproduce the error:
antlr4 -Dlanguage=Python3 PythonLexer.g4
antlr4 -Dlanguage=Python3 PythonParser.g4
pygrun --tokens Python file_input fractions.py
error message:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 15765: ordinal not in range(128)
I found the easiest way to reproduce the error.
For example the following 2-character Python code (which is a comment) causes the same error message:
#Å
More precisely: Unicode Character “Å” (U+00C5)