Reapply "Switch from json11 to nlohmann/json (#31093)"

[adeebshihadeh]

[mtribiere]

Hey, I saw you reverted the commit, is there anything wrong with it? May I help?

[jnewb1]

Hey, I saw you reverted the commit, is there anything wrong with it? May I help?

there is something wrong with the thneed part, it crashes on device. If you have a device you can probably test it there. If not I'm planning to look into it today.

modeld crashes with: pytest selfdrive/test/test_onroad.py -s

[mtribiere]

I sadly do not possess hardware, that's why I chose this bounty PR initially. If there is something else I can help, without hardware let me know.

[adeebshihadeh]

I sadly do not possess hardware, that's why I chose this bounty PR initially. If there is something else I can help, without hardware let me know.

@mtribiere you can repro this by running this in selfdrive/modeld/: scons -u -j8 --pc-thneed && USE_THNEED=1 ./modeld

batman:modeld$ scons -u -j8 --pc-thneed && USE_THNEED=1 ./modeld
scons: Entering directory `/home/batman/fivepilot'
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
scons: `selfdrive/modeld' is up to date.
scons: done building targets.
Thneed::clinit done
Thneed::load: loading from /home/batman/fivepilot/selfdrive/modeld/models/supercombo.thneed
terminate called after throwing an instance of 'nlohmann::json_abi_v3_11_3::detail::parse_error'
  what():  [json.exception.parse_error.101] parse error at line 1, column 19981: syntax error while parsing value - invalid string: ill-formed UTF-8 byte; last read: '"'
Aborted (core dumped)

[mtribiere]

Seems like this library only support UTF-8 encoded string, that's probably why the parse function fails.

Only UTF-8 encoded input is supported which is the default encoding for JSON according to RFC 8259.

[adeebshihadeh]

Seems like this library only support UTF-8 encoded string, that's probably why the parse function fails.

Only UTF-8 encoded input is supported which is the default encoding for JSON according to RFC 8259.

I see. Any way around it? Or we'll have to go with another library?

[mtribiere]

A way around it, probably not as it's how the library is built. The only solution I see right now is to modify how this JSON is produced in the first place. Is that a viable option for you? Might require some additional work though

[adeebshihadeh]

Yes, that should be fine.