bitmath icon indicating copy to clipboard operation
bitmath copied to clipboard

Comparison issues with LARGE units

Open tbielawa opened this issue 8 years ago • 0 comments

I began testing out adding support for the ZiB, YiB, Zib, and Yib units while working on #53. This broke the (new) unit tests immediately and in a non-obvious way. For example, when parsing 654 Zo:

======================================================================
FAIL: parse_string works on zettaoctet strings
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tbielawa/Projects/bitmath/tests/test_parse.py", line 87, in test_parse_Zo
    compare_to)
AssertionError: ZB(654.0) != ZB(654.0)

Interesting I think that the error message says two equal things are not equal, ZB(654.0) != ZB(654.0).

I did some initial debugging and figured out how the comparison is failing. It appears that the instances bytes parameters are of different types!

  • Parsed input bytes: 6.54e+23
  • Comparson bytes: 654000000000000000000000

Throw this into python real quck:

In [1]: n1 = 6.54e+23

In [2]: n2 = 654000000000000000000000

In [3]: print type(n1), type(n2)
<type 'float'> <type 'long'>

In [4]: n1 == n2
Out[4]: False

In [5]: n1 - n2
Out[5]: 0.0

BUT, you can work around this issue by explicitly instantiating the 'to parse' and the 'comparison' instances as floating point numbers

In [1]: import bitmath

In [2]: parsed = bitmath.parse_string("654.0 Zo")

In [4]: compare_to = bitmath.ZB(654.0)

In [5]: parsed == compare_to
Out[5]: True

In [8]: print parsed.bytes, compare_to.bytes
6.54e+23 6.54e+23

In [9]: print type(parsed.bytes), type(compare_to.bytes)
<type 'float'> <type 'float'>

For now I'm going to remove the code I added for the new unit support and then revisit this after finishing up the work in #53

tbielawa avatar Dec 08 '15 16:12 tbielawa