DB48X-on-DM42 icon indicating copy to clipboard operation
DB48X-on-DM42 copied to clipboard

Combining `FromHMS` and `ToHMS` gives unrounded result

Open c3d opened this issue 6 months ago • 2 comments
trafficstars

Running 10.3033 FromHMS ToHMS gives the result 10.30.32.999629... instead of the expected 10.30.33.

Here is the original report

c3d avatar May 07 '25 20:05 c3d

Problem is caused by undesired conversion to fraction. In that specific case, the generated fraction is not generated with sufficient precision.

c3d avatar May 07 '25 21:05 c3d

The correct solution is to make sure that the default value for FractionIterations is big enough to give us the correct result.

c3d avatar May 07 '25 21:05 c3d