leveldb-ruby icon indicating copy to clipboard operation
leveldb-ruby copied to clipboard

Fails when writing

Open xumingyong opened this issue 13 years ago • 7 comments

Code as following 100_000.times { |x| db.put x.to_s, "abcdefghijklmnopqrstuvwxyz"}

Error: .../lib/leveldb/leveldb.so: undefined symbol: _ZN6snappy19MaxCompressedLengthEj when more than about 79000 keys.

DB folder shows: -rw-r--r-- 1 root root 4178034 2012-04-18 07:58 000003.log -rw-r--r-- 1 root root 65536 2012-04-18 07:58 000004.log -rw-r--r-- 1 root root 0 2012-04-18 07:58 000005.sst -rw-r--r-- 1 root root 16 2012-04-18 07:58 CURRENT -rw-r--r-- 1 root root 0 2012-04-18 07:58 LOCK -rw-r--r-- 1 root root 115 2012-04-18 07:58 LOG -rw-r--r-- 1 root root 65536 2012-04-18 07:58 MANIFEST-000002

Why ???

xumingyong avatar Apr 18 '12 07:04 xumingyong

I guess the reason is:

the default write_buffer_size is 4MB, so when the Memory flush to disk, LevelDB call snappy function, but the LevelDB-ruby did not include the snappy lib.

xumingyong avatar Apr 18 '12 16:04 xumingyong

It solved.

  1. download leveldb-ruby source code.
  2. modify the line12 of ./ext/leveldb/extconf.rb from $LIBS << " -L../../leveldb -lleveldb" to $LIBS << " -L../../leveldb -lleveldb -lsnappy"
  3. run # rake -T spec # ruby setup.rb # cd /usr/local/lib/site-ruby/1.9.1/i686-linux/ # cp leveldb.so ../leveldb/
  4. copy snappy.so.1 to above leveldb path.
  5. try the test code.

xumingyong avatar Apr 19 '12 07:04 xumingyong

Hello,

Thanks for the report. But I can't reproduce this. What version of the leveldb-ruby gem do you have installed?

wmorgan avatar Apr 20 '12 21:04 wmorgan

The gem version is 5975fd7

My test enviroment is UNBUNTU 11.10

xumingyong avatar Apr 21 '12 02:04 xumingyong

Can you try the latest master? I've updated to the latest leveldb code and this solved the snappy problem for some people (e.g. https://github.com/wmorgan/leveldb-ruby/pull/20)

wmorgan avatar Apr 22 '12 21:04 wmorgan

1. Try the new rubyforge gem

I tried the :

gem install level-ruby

again, something make me confused, the ruby report in put: wrong argument type LevelDB::DB (expected Data) (TypeError)

but your test file works.

2. Try the latest master, and build it manually

commands:

rake gem setup.ruby cp ./i686-linux/leveldb.so ./leveldb/

.../lib/leveldb/leveldb.so: undefined symbol: _ZN6snappy19MaxCompressedLengthEj

It seems it does not work, I guess the g++ command need -lsnappy switch in Makefile of google leveldb source files.

xumingyong avatar Apr 23 '12 05:04 xumingyong

I don't know if you're still experiencing this problem, but I solved this by just remove snappy/leveldb from my package manager, then reinstalled the gem.

emnl avatar Nov 04 '12 19:11 emnl