crc32 is overridden by another library when using bundler

[n-nishizawa]

I inserted the same input, but I got different bitmap on Linux and Mac.

I ran the following code.

• test.rb

require 'cbloomfilter'

bf = CBloomFilter.new(16, 1, 7, 1)

puts "m #{bf.m}"
puts "k #{bf.k}"
puts "b #{bf.b}"

bf.insert 'a'
puts "bitmap: #{bf}"

• Result on Linux

$ bundle exec ruby test.rb
m 16
k 1
b 1
bitmap: 1000000000000000

• Result on Mac

m 16
k 1
b 1
bitmap: 0000000000000100

I added rb_warn into crc32 to check if the function crc32 in your gem is called.

... snip ...

#include "crc32.h"

unsigned int crc32(unsigned int crc, char *buf, int len) {
    while (len > 0) {
        crc = crc_table[(crc ^ *buf) & 0xff] ^ (crc >> 8);
        rb_warn("crc: %d\n", crc);
        --len;
        ++buf;
    }
    return crc;
}

And I ran test.rb with bundler, but I got no warning output by rb_warn. From this result, maybe crc32 in your gem is not called.

I also ran test.rb without bundler specifying LOAD_PATH explicitly.

$ ruby -I./vendor/bundle/ruby/2.2.0/bundler/gems/bloomfilter-rb-8c9da9516796/ext/cbloomfilter test.rb
m 16
k 1
b 1
test.rb:9: warning: crc: -1529756563

bitmap: 0000000000000100

The waning output came out and I got the same bitmap as that on Mac.

So crc32 seems to be overridden by another library when using bundler on Linux.

[igrigorik]

Huh, well that's a fun one. Thanks for tracking this down... /me scratches head, not quite sure what's going here.

I guess we could/should the built-in method: http://ruby-doc.org/stdlib-1.9.3/libdoc/zlib/rdoc/Zlib.html#method-c-crc32

[n-nishizawa]

Thank you so much for your reply!!

I have no idea why this strange thing happens too...

In my investigation, Zlib.crc32 might be used on Linux with bundler. Because the index based on Zlib.crc32 corresponds to this gems's bit field.

[6] pry(main)> require 'cbloomfilter'
=> true
[7] pry(main)>
[8] pry(main)>
[9] pry(main)>
[10] pry(main)> bf = CBloomFilter.new(16, 1, 7, 1)
=> #<CBloomFilter:0x007f66e7290cc0 @hash_value={}>
[11] pry(main)>
[12] pry(main)>
[13] pry(main)>
[14] pry(main)>
[15] pry(main)> bf.insert 'a'
=> nil
[16] pry(main)>
[17] pry(main)> bf.to_s
=> "1000000000000000"
[18] pry(main)>
[19] pry(main)>
[20] pry(main)> Zlib.crc32('a', 7) % 16
=> 0
[21] pry(main)>
[22] pry(main)> bf.insert 'b'
=> nil
[23] pry(main)>
[24] pry(main)> bf.to_s
=> "1000000000100000"
[25] pry(main)>
[26] pry(main)> Zlib.crc32('b', 7) % 16
=> 10
[27] pry(main)>

To fix it, I think there are two options. One is to rename crc32 in your gem to prevent the name conflict. Another is to use builtin crc32 such as Zlib.crc32 like your suggestion.

[igrigorik]

We should update the code to use the provided crc32 implementation.. no reason to duplicate it.