Always use `encoding="utf-8-sig"` when reading text files
Changed the encoding from utf8 to utf-8-sig when reading files, in order to ignore a possible byte-order mark (a.k.a. BOM, code point U+FEFF) at the start of the file.
As per the Python documentation:
In some areas, it is also convention to use a “BOM” at the start of UTF-8 encoded files; the name is misleading since UTF-8 is not byte-order dependent. The mark simply announces that the file is encoded in UTF-8. For reading such files, use the ‘utf-8-sig’ codec to automatically skip the mark if present.
https://docs.python.org/3/howto/unicode.html#reading-and-writing-unicode-data
So this change won't affect reading UTF8-encoded files without a BOM.
Fixes #386.
I don't think this PR has anything to do with the reported Mypy errors, unless I'm missing something. (I only ran pytest before submitting.)
Thanks! You can rebase on main to get rid of the mypy warnings :+1:
Oh, can you please add a test that runs on Windows only, asserting the fix works? It should check that trying to load a BOM'd module with UTF8 raises a LoadingError, while it works with UTF8-SIG.