Tomlyn icon indicating copy to clipboard operation
Tomlyn copied to clipboard

Not serializing floats properly

Open snarlynarwhal opened this issue 2 years ago • 2 comments

It seems like floats < 1.0 get serialized weird. For example, .1 gets serialized as 0.100000001 and .99 gets serialized as 0.99000001.

snarlynarwhal avatar Mar 27 '23 17:03 snarlynarwhal

I wouldn't call this a Tomlyn bug - it's getting 0.100000001 from your code since that's what a 32-bit floating point is stored as in memory (well base 2, but the equivalent is that). You could switch to double or decimal depending on your use case to reduce imprecision, but I don't think Tomlyn should change the value it's getting based on assumptions about the input being base 10 and the user's aesthetic preferences. A custom converter is probably where any such logic could stay, in my opinion.

lilith avatar Jan 31 '24 22:01 lilith

I apologize, it looks like it may be possible to improve the situation, although I'm uncertain if we can do so with the built-in formatters in .NET. https://github.com/xoofx/Tomlyn/pull/81/commits/026f953e5f3647f0ec83d3ad3fd2fab406261bad

lilith avatar Feb 01 '24 04:02 lilith