jsonnet
jsonnet copied to clipboard
google
Numbers should manifest with same string representation as other programming environments
$ cat foo.jsonnet
0.9
$ jsonnet foo.jsonnet
0.90000000000000002
Not sure if there's a formal standard, but everyone else seems to agree on printing it one particular way, so maybe just follow that:
$ echo "print 0.9" | python
0.9
$ echo "console.log(0.9)" | node
0.9
$ echo "println(0.9)" | scala
0.9
You're manifesting a string, if you do std.manifestJson(0.9) you'll get the same result.
As for whether it should actually print 0.9, I think there is a case for that because 0.9 is just as accurate because it will round to 0.90000000000000002.
I'm not sure quite how to achieve that though, the current implementations are: https://github.com/google/jsonnet/blob/master/core/parser.cpp#L34 https://github.com/google/go-jsonnet/blob/master/interpreter.go#L602
Good point, my example is incorrect. It still would be nice to print the same thing as everyone else though
It would be easy to just keep chopping chars off the end until the reparsing of the string yielded a different number. However I'm hoping there is a better way.
Printing floating-point numbers require a bit of thinking.
- Printing Floating-Point Numbers (and references therein)
- Printing Floating-Point Numbers - An Always Correct Method
maybe we should have a std.printf
This is coming in C++17 via std::to_chars
http://eel.is/c++draft/charconv.to.chars#2
There are 3rd party algorithms we could include in the meantime, e.g. grisu3 which can be found included in many math packages but doesn't seem to have an official source. The paper linked above is better than grisu3 and its implementation is at http://goto.ucsd.edu/~andrysco/errol/
I haven't looked into what golang can do.
I noticed while reviewing #662 that json.hpp which we already depend on now has an implementation of a suitable algorithm for doing this.
That's great. Could you point me to where exactly?
I wonder how difficult it would be to reproduce this behavior exactly in Go (and ideally specify it precisely enough to make it possible for sjsonnet folks to be fully compatible as well).
https://github.com/google/jsonnet/blob/master/third_party/json/json.hpp#L11957
This seems to do the right thing in Go:
fmt.Println(strconv.FormatFloat(0.9, 'g', -1, 64))
The precision behavior is quite easy to specify -- It should produce the shortest string that still parses back into the original value. However the point at which it decides to use exponential notation might differ between implementations.
In the case of jsonnet, the string representation of the decimal should be the same as the string in the jsonnet... If it's not evaluating an expression, it shouldn't be changing the data! I posted an equivalent issue before being led here: https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/jsonnet/utRE84lrDRQ/MAGIh_yeBwAJ
It never changes the actual value. The problem is that 0.9 and 0.90000000000000002 are actually the same number in double precision floating point. Also 9E-1. So the point is to pick the most human readable of all the possibilities. While we could try to preserve the original formatting (so e.g. 90E-2 would come out as 90E-2 instead of 0.9) this would be tricky because where do you draw the line and say this is actually evaluating an expression? Assigning it to a variable / field / array element? Passing it through a function? Adding 0 to it? Better just to print 90E-2 as 0.9 in all cases so it's simple to understand what's going on.
OK, this bug is a different issue to the one I'm pointing out in my link: When JSonnet parses the text representation of an immediate double it should optimally know that it is not an expression and, therefore, requires no evaluation. The JSON text representation of the double should be identical to the JSonnet text representation of the double (just copy the text)!
This thread is focusing on how best to represent the result of an expression resulting in a decimal result which is an entirely different code area... Personally, I would note whether scientific notation had ever been used in that particular JSonnet file before and only output using Scientific notation if it had... But I'm sure a more attractive solution must exist like allowing a formatting string after an expression - which could allow more interesting results like sting hexadecimal output for integers etc. but I'll get back to my own thread before I cause trouble... ;-)
Is there any workaround available to make floating numbers work as described in the OP by @lihaoyi-databricks ?
At this point, no. I think the blocker here is to specify when to use exponential (scientific notation). If should depend only on the number (in particular we don't want to depend on how it's formatted in the source file - too magical).
If you have any ideas, please let us know!
We can probably just pick something and all implementations can use the same threshold.
@sparkprime's suggestion seems to do the trick:
fmt.Println(strconv.FormatFloat(0.9, 'g', -1, 64))
Though it returns a string and I'm not familiar how go-jsonnet will then make sure the value doesn't receive string quotes in the JSON output.
Maybe adding another function to the Standard Library, e.g. std.parseDecimal(str, precision), to support specified precision leaving the current default behavior untouched.
That's not a problem, it returns a string that is compatible with a JSON number.
Just FYI, in a statically linked build (see below) the default float formatting behavior appears to have changed in std.manifestJson and std.manifestJsonEx with 0.17.
Version 0.16.0:
$ jsonnet -e '0.9'
0.90000000000000002
$ jsonnet -e 'std.manifestJson({'test':0.9})'
"{\n \"test\": 0.90000000000000002\n}"
$ jsonnet -e 'std.manifestJsonEx({'test':0.9},"")'
"{\n\"test\": 0.90000000000000002\n}"
Version 0.17.0:
$ jsonnet -e '0.9'
0.90000000000000002
$ jsonnet -e 'std.manifestJson({'test':0.9})'
"{\n \"test\": 0.9\n}"
$ jsonnet -e 'std.manifestJsonEx({'test':0.9},"")'
"{\n\"test\": 0.9\n}"
This was tested with the statically linked jsonnet from rpm golang-github-google-jsonnet on Fedora 33. However, using the binary from https://github.com/google/jsonnet/releases/download/v0.17.0/jsonnet-bin-v0.17.0-linux.tar.gz (linked to libstdc++, among others) results in the old behavior.
