sext icon indicating copy to clipboard operation
sext copied to clipboard

Broken decoding of big positive floats

Open seriyps opened this issue 1 year ago • 1 comments

Maybe I'm doing smth wrong but:

> sext:encode(2200000000.1).
<<11,255,193,48,114,26,180,0,8,1,140,230,115,0,5>>
> sext:decode(sext:encode(2200000000.1)).
** exception error: no match of right hand side value <<144,171,0,12,204,51:6>>
     in function  sext:decode_pos/3 (/Users/***l/_build/test/lib/sext/src/sext.erl, line 987)
     in call from sext:decode/1 (/Users/***/_build/test/lib/sext/src/sext.erl, line 238)

However it works for big negative floats:

> sext:encode(-2200000000.1).
<<8,255,255,255,254,255,194,96,31,255,255,255,255,124,239,
  106,127,224,8,0,115,25,140,255,250>>
> sext:decode(sext:encode(-2200000000.1)).
-2200000000.1

seriyps avatar Oct 14 '24 16:10 seriyps

Apologies. I've had a series of really busy weeks. Will try to get to analyzing this soon.

uwiger avatar Oct 25 '24 07:10 uwiger