middleware icon indicating copy to clipboard operation
middleware copied to clipboard

NAS-116747 / 22.12 / Provide string values for JSON fields that may overflow 2**53 on PiB-scale systems

Open themylogin opened this issue 3 years ago • 5 comments

Some JSON parsers (e.g. the one that is used in the jq utility for example) don't parse large values correctly, as a result we have this:

      "snapshot_limit": {
        "parsed": 18446744073709552000,
        "rawvalue": "18446744073709551615",
        "value": "none",
        "source": "DEFAULT",
        "source_info": null
      },

(the middleware JSON output was correct, but after going through jq the parsed and rawvalue become different).

Clients that know that their JSON parsing is poor can restore the correct ZFS property value using rawvalue which is always a string. For other JSON dictionaries that might contain integer values that are too large (this might happen when we start selling petabyte-scale systems) we'll provide f'{key}_str values like many other REST APIs do (e.g. Twitter tweet IDs, etc). These are all such values that I was able to find.

themylogin avatar Aug 10 '22 18:08 themylogin

Jira URL: https://ixsystems.atlassian.net/browse/NAS-116747

bugclerk avatar Aug 10 '22 18:08 bugclerk

time 4:00

themylogin avatar Aug 10 '22 18:08 themylogin

Are there any tests that need to be updated?

anodos325 avatar Aug 11 '22 17:08 anodos325

@anodos325 I'll ran the tests once they fix CI. I have not found any that will fail due to extra keys.

themylogin avatar Aug 11 '22 18:08 themylogin

@anodos325 tests are fine, no regressions

themylogin avatar Aug 12 '22 08:08 themylogin

@anodos325 are we good to merge this?

themylogin avatar Aug 15 '22 09:08 themylogin