conjure icon indicating copy to clipboard operation
conjure copied to clipboard

Clarify double/lossiness handling

Open raiju opened this issue 4 years ago • 2 comments

What happened?

Doubles in conjure-java failed for some numbers that should've been supported: https://github.com/palantir/conjure-java/pull/1006. While discussing the fix we realized we haven't defined how we expect conjure to handle doubles, specifically around lossiness in deserialization.

What did you want to happen?

The following questions should be answered:

  1. Should the deserialization behavior of an integral JSON number (\d+) be identical to a floating point JSON number w/ a zero decimal value (\d+\.0)? E.g. should 9007199254740993 result in the same behavior as 9007199254740993.0?
  2. Are any "losses" acceptable? Examples (received value, returned value)
  • 9007199254740993 -> 9007199254740992.0
  • 9007199254740993.0 -> 9007199254740992.0
  • 9007199254740992.1 -> 9007199254740992.0
  • 0.30000000000000004 -> 0.30000000000000004
  • 0.30000000000000003 -> 0.30000000000000004
  • 0.30000000000000002 -> 0.30000000000000004
  • 0.30000000000000001 -> 0.3

Note that these results are a bit misleading. Floating points technically don't actually describe a single value, but a range (e.g. 0.3 is actually 0.29999999999999996... - 0.30000000000000001..., the center point is ~ 0.29999999999999998; when a double gets printed, it'll emit the number in that range with the shortest decimal representation; this is also the cause of the classic 0.1 + 0.2 = 0.30000000000000004).

raiju avatar Jul 13 '20 19:07 raiju

My proposal is to:

  1. Treat \d+\.0 and \d+ identically (i.e. do not special-case integral inputs)
  2. Treat doubles as lossy/imprecise. If exact transferring of numbers is required, integer, long or decimal types should be used.

raiju avatar Jul 13 '20 19:07 raiju

@carterkozak @markelliot (as you both had opinions on the conjure-java PR)

raiju avatar Jul 13 '20 20:07 raiju