msgpack-perl icon indicating copy to clipboard operation
msgpack-perl copied to clipboard

Should prefer IV when both IV and NV are OK

Open vphantom opened this issue 6 years ago • 1 comments

Hello,

After a lot of digging, I finally managed to understand why this module in my test script was always assuming that integers were huge float64's. Here's my discovery:

use Devel::Peek;
use Data::MessagePack;
my $mp = new Data::MessagePack;
$mp->prefer_integer(1);

my $a = [ 30000, 123, 300, 1000, 1000, 500, 500 ];

print 'Clean: ', unpack('H*', $mp->pack($a)), "\n";
#print Dump($a);
my $b = pack('wwwwwww', @{ $a });
#print Dump($a);
print 'After pack(): ', unpack('H*', $mp->pack($a)), "\n";

This outputs:

Clean: 97cd75307bcd012ccd03e8cd03e8cd01f4cd01f4
After pack(): 97cb40dd4c0000000000cb405ec00000000000cb4072c00000000000cb408f400000000000cb408f400000000000cb407f400000000000cb407f400000000000

Using Devel::Peek's Dump() I finally figured out that pack() does its reading in a way which triggers Perl to add an NV in addition to the original IV. Before pack() the first value is:

SV = IV(0x17d32b0) at 0x17d32c0
  REFCNT = 1
  FLAGS = (PADMY,IOK,pIOK)
  IV = 30000

But after pack() reads it, it becomes:

SV = PVNV(0x1934710) at 0x17d32c0
  REFCNT = 1
  FLAGS = (PADMY,IOK,NOK,pIOK,pNOK)
  IV = 30000
  NV = 30000
  PV = 0

My understanding is that the integer version of the value is still usable, but MessagePack prioritizes the float version. My first attempt at fixing that (PP.pm#274 and pack.c#193) broke test 21 "dirty float" so there seems to be more to Perl's IOK/NOK than meets my eye.

I'll experiment further and hopefully come up with a pull request. I figure there must be an efficient way to determine whether the IV loses precision vs the NV. If it doesn't then IV wins, if it does then NV wins.

vphantom avatar Jan 11 '18 14:01 vphantom