nix icon indicating copy to clipboard operation
nix copied to clipboard

fix precision loss when converting floating point numbers to strings

Open pennae opened this issue 2 years ago • 4 comments

currently floating point conversions are done with 6 digits of precision, which is adequate for float but can lose precision for double. toString additionally uses the fixed format rather than the general format, so it will truncate numbers to 0 that have an absolute value less than 5e-7.

fixes #5733

pennae avatar Mar 12 '22 01:03 pennae

This changes evaluation results, which is potentially a big problem...

edolstra avatar Mar 12 '22 08:03 edolstra

yeah, that is a potential problem :confused: there are a few nixos modules that use floats, but those shouldn't be impacted too much because upgrading nix would most likely rebuild service definitions anyway. in nixpkgs packages themselves floats seem to be pretty rare, running the ofborg outpaths check over all of nixpkgs calls Value::mkFloat only 29 times. scientific workloads using nix for reproducible data collection and analysis could benefit from better precision though.

the other option would be to prominently document that nix will not output floats with more than 6 fractional digits, ever, unless someone adds a builtin that can do that (which would likely involve versions of toJSON and toXML as well). it's all a bit messy :slightly_frowning_face:

pennae avatar Mar 12 '22 16:03 pennae

this would be really really nice to have

I was very confused earlier by nix telling me that 20.350000 and 20.350000 were not equal; the only way I worked out to see the actual value was to multiply by 1000000000 before printing, at which point I got to see that the first value is actually something like 20.349999999999996

sersorrel avatar Jun 08 '22 19:06 sersorrel

I think for now we could at least introduce a builtins.floatToString that uses Ryū as suggested by @andersk, so that we don't need to change the evaluation result but at least have an alternate way to print floats correctly. Eventually we could deprecate toString for floats to remove the buggy behavior.

infinisil avatar Sep 16 '22 13:09 infinisil