BreakInfinity.cs icon indicating copy to clipboard operation
BreakInfinity.cs copied to clipboard

Exponent might be wrong

Open Khaeops opened this issue 2 years ago • 1 comments

While trying out this package I noticed that the exponents don't seem to be working as expected. Typing '1000' into the left field in the Unity Inspector yields a value of 1e6, and typing '1000000' (1 million) yields a value of 1e21. I would imagine the expected values should be 1e3 and 1e6, respectively.

Khaeops avatar Oct 23 '23 12:10 Khaeops

That is due to the way the custom drawer for BigDouble works. Its value is updated every time the field content changes. For example, when you type in 1000 into the mantissa field, this is what happens:

  1. Type in 1.
  2. Field is 1.
  3. Value is {mantissa: 1, exponent: 0} and displayed as 1.
  4. Type in 0.
  5. Field is 10.
  6. Value is {mantissa: 10, exponent: 0}.
  7. Value is normalized to {mantissa: 1, exponent: 1} and displayed as 10.
  8. Type in 0.
  9. Field is 100.
  10. Value is {mantissa: 100, exponent: 1}.
  11. Value is normalized to {mantissa: 1, exponent: 3} and displayed as 1000.
  12. Type in 0.
  13. Field is 1000.
  14. Value is {mantissa: 1000, exponent 3}.
  15. Value is normalized to {mantissa: 1, exponent 6} and displayed as 1000000.

So, this is a problem in the custom drawer instead of BigDouble itself.

asdfchlwnsgy1236 avatar Jul 13 '24 08:07 asdfchlwnsgy1236