snarkOS icon indicating copy to clipboard operation
snarkOS copied to clipboard

`u64` field should be serialized as string in json format

Open HarukaMa opened this issue 2 years ago • 7 comments

🐛 Bug Report

The json formatter is storing u64 fields as a number, and it's silently losing some precision for the data. I'm not sure why it happened during the serialization though - I'd expect rust libraries use u64 instead of double for that.

Currently the only place affected is the nonce of the PartialSolution.

Steps to Reproduce

REST endpoint testnet3/latest/block reports inaccurate nonce for coinbase solutions.

Expected Behavior

The data should either be accurate numbers or strings.

Your Environment

Assume latest

HarukaMa avatar Nov 10 '22 07:11 HarukaMa

Thank you for reporting this issue @HarukaMa. This is puzzling, the Serializer implementation work fine with serde's JSON, as tested in snarkVM for PartialSolution; and this trait is simply what warp uses under the hood in order to handle the data. What are you using to query the rest API? 🤔

niklaslong avatar Nov 10 '22 10:11 niklaslong

wow, this might be an invalid issue then. It looks like jq is actually the culprit here. The snarkOS code is actually producing good results. That's a little embarrassing.

That said, I've seen another consumer of the rest api incorrectly dealing with those large numbers. Maybe using strings could raise the awareness about that?


To clarify, there is another block explorer outta there and it's also losing precision on the nonce field. The data in the current form is not safe in the javascript environment, as js uses double for all numbers.

HarukaMa avatar Nov 10 '22 10:11 HarukaMa

I see, good to know and agreed; we'll definitely consider the switch and if we don't end up making the change we'll document this better. Thanks again!

niklaslong avatar Nov 10 '22 10:11 niklaslong

yeah, jq handles numbers only to about 2^53 which is mantissa size in double precision floats. You could try using jql instead of jq, it will handle u64 and probably also even u128

zvolin avatar Nov 10 '22 11:11 zvolin

please note that jql will probably be a lot slower than jq when reading from stdin (at least that was true a month ago when I was checking). It prefers files provided as an argument

zvolin avatar Nov 10 '22 11:11 zvolin

Check in here, is this issue still relevant?

howardwu avatar Oct 09 '23 23:10 howardwu

Yes, the nonce is still returned in json as a number.

HarukaMa avatar Oct 10 '23 19:10 HarukaMa