ruff icon indicating copy to clipboard operation
ruff copied to clipboard

UTF8 files with BOM produce E999 "unexpected token"

Open danielmenzel opened this issue 3 years ago • 2 comments

I tried using ruff 0.0.191 (Python 3.9 on Windows 10) to check a file that is encoded in UTF8 with BOM (byte order mark). I used the default options for ruff.

Ruff reported a E999 SyntaxError: Got unexpected token in line 1 column 2 (and then apparently stopped processing further rules except for line length warnings).

When I changed the encoding of the file to UTF8 without BOM, ruff worked as expected and no E999 was reported.

As a workaround I can just change the encoding of the files to make sure no BOM is present, but I think it would be nice if ruff would be able to handle any of the usual source encodings.

danielmenzel avatar Dec 22 '22 20:12 danielmenzel

Yeah definitely a bug!

charliermarsh avatar Dec 23 '22 03:12 charliermarsh

I've raised https://github.com/RustPython/RustPython/issues/4353 for this.

squiddy avatar Dec 23 '22 19:12 squiddy