subway icon indicating copy to clipboard operation
subway copied to clipboard

Saved JSON has a byte order mark (BOM)

Open Kenny2github opened this issue 3 years ago • 0 comments

Hexdump of the first 16 bytes of a JSON file saved from https://jpwright.github.io/subway is as follows:

$ hd -n 16 bns_saved_game.json
00000000  ef bb bf 7b 22 6c 69 6e  65 73 22 3a 5b 7b 22 6e  |...{"lines":[{"n|
00000010

The first three bytes of the file, ef bb bf, are the UTF-8 BOM, and when trying to parse the JSON via other means, result in the JSON being treated as invalid:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
    json.load(f)
  File "...\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 293, in load
    return loads(fp.read(),
  File "...\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "...\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "...\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

(Note that Python 3.5 on Debian WSL is more helpful in its erroring:)

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
    json.load(f)
  File "/usr/lib/python3.5/json/__init__.py", line 268, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/usr/lib/python3.5/json/__init__.py", line 315, in loads
    s, 0)
json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)

This BOM has no good reason to be there, as modern editors (even including Notepad) can detect UTF-8 text without it. It should be removed.

Kenny2github avatar Jul 29 '21 17:07 Kenny2github