DB48X-on-DM42 icon indicating copy to clipboard operation
DB48X-on-DM42 copied to clipboard

Converting HwFP to fraction gives truncated result

Open c3d opened this issue 11 months ago • 1 comments

Running the following gives a totall wrong result:

HFP 12 PREC
10.3033 FromHMS ToHMS

We expect 10.30.33, instead we get 10:00:00

Initial report

c3d avatar May 07 '25 21:05 c3d

Caused by an incorrect use of - operator on uint value, which causes the epsilon to be very large instead of very small.

c3d avatar May 07 '25 21:05 c3d