ctypes.sh icon indicating copy to clipboard operation
ctypes.sh copied to clipboard

Precision of float/double values during output

Open Hunter-Github opened this issue 10 years ago • 5 comments

This is a wonderful library, am trying it now, and have noticed that double values are truncated at output to 6 digits.

A possible solution could be adding a printf format specifier in parens after the type argument:

-r double(.12f)

Would be awesome to be able to print exact hexadecimal contents of the returned value since most IEEE 754 to decimal string conversions will lose information.

Hunter-Github avatar Aug 02 '15 20:08 Hunter-Github

Hmm, good idea. I'll try to prototype some code today.

taviso avatar Aug 03 '15 18:08 taviso

I think writing it in hex is the correct solution, but doing it by default makes it harder to interact with the numbers in bash. What about having a rawdouble:/rawfloat: type for when you don't want to lose precision?

taviso avatar Aug 03 '15 18:08 taviso

Fine by me, thanks!

Hunter-Github avatar Aug 03 '15 20:08 Hunter-Github

Hm, not sure why Github didn't auto-tag this issue from b862cd0d1753cb0bcab35088b672a5ce3bf74c4b. It's related. Maybe sufficient for your needs? Did you have something else in mind, or are you aware of any way it is deficient? Thanks.

cemeyer avatar Aug 14 '15 22:08 cemeyer

Thanks, looks good to me, I can imagine how someone wants to chain a bunch of libm operations without losing precision until the final operation...perfect.

taviso avatar Aug 15 '15 14:08 taviso