pyrocksdb icon indicating copy to clipboard operation
pyrocksdb copied to clipboard

Published 20 hours ago • verified publisher icon stephan-hof

pyrocksdb install issue

Open abuvaneswari opened this issue 7 years ago • 2 comments

Hello,

Please help. I just installed rocksdb and followed instructions in your pyrocksdb page (and consulted with http://www.guguncube.com/2925/rocksdb-installation-and-performance-tests as well). I am getting the following error.

Please help me fix it.

Platform version: Ubuntu 14.04

thanks, Buvana

Error message follows-----

(pyrocks_test):~/rocksdb/rocksdb/pyrocks_test$ pip install pyrocksdb Downloading/unpacking pyrocksdb Downloading pyrocksdb-0.4.tar.gz (139kB): 139kB downloaded Running setup.py (path:/nas1/buvana/rocksdb/rocksdb/pyrocks_test/build/pyrocksdb/setup.py) egg_info for package pyrocksdb

Requirement already satisfied (use --upgrade to upgrade): setuptools in ./lib/python2.7/site-packages (from pyrocksdb) Installing collected packages: pyrocksdb Running setup.py install for pyrocksdb

building 'rocksdb._rocksdb' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing
cc1plus: warning: command line option \u2018-Wstrict-prototypes\u2019 is valid for C/ObjC but not for C++ [enabled by default]
rocksdb/_rocksdb.cpp: In function \u2018PyObject* __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_4create_filter(__pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy*, PyObject*)\u2019:
rocksdb/_rocksdb.cpp:5639:88: warning: conversion to \u2018int\u2019 from \u2018std::vector<rocksdb::Slice>::size_type {aka long unsigned int}\u2019 may alter its value [-Wconversion]
     __pyx_t_6->CreateFilter(py_rocks::vector_data(__pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst));
                                                                                        ^
rocksdb/_rocksdb.cpp: In function \u2018PyObject* __pyx_pf_7rocksdb_8_rocksdb_7Options_26expanded_compaction_factor___get__(__pyx_obj_7rocksdb_8_rocksdb_Options*)\u2019:
rocksdb/_rocksdb.cpp:13229:56: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018expanded_compaction_factor\u2019
   __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->opts->expanded_compaction_factor); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 873; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                        ^
rocksdb/_rocksdb.cpp: In function \u2018int __pyx_pf_7rocksdb_8_rocksdb_7Options_26expanded_compaction_factor_2__set__(__pyx_obj_7rocksdb_8_rocksdb_Options*, PyObject*)\u2019:
rocksdb/_rocksdb.cpp:13292:23: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018expanded_compaction_factor\u2019
   __pyx_v_self->opts->expanded_compaction_factor = __pyx_t_1;
                       ^
rocksdb/_rocksdb.cpp: In function \u2018PyObject* __pyx_pf_7rocksdb_8_rocksdb_7Options_24source_compaction_factor___get__(__pyx_obj_7rocksdb_8_rocksdb_Options*)\u2019:
rocksdb/_rocksdb.cpp:13351:56: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018source_compaction_factor\u2019
   __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->opts->source_compaction_factor); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 879; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                        ^
rocksdb/_rocksdb.cpp: In function \u2018int __pyx_pf_7rocksdb_8_rocksdb_7Options_24source_compaction_factor_2__set__(__pyx_obj_7rocksdb_8_rocksdb_Options*, PyObject*)\u2019:
rocksdb/_rocksdb.cpp:13414:23: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018source_compaction_factor\u2019
   __pyx_v_self->opts->source_compaction_factor = __pyx_t_1;
                       ^
rocksdb/_rocksdb.cpp: In function \u2018PyObject* __pyx_pf_7rocksdb_8_rocksdb_7Options_30max_grandparent_overlap_factor___get__(__pyx_obj_7rocksdb_8_rocksdb_Options*)\u2019:
rocksdb/_rocksdb.cpp:13473:56: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018max_grandparent_overlap_factor\u2019
   __pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->opts->max_grandparent_overlap_factor); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 885; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                        ^
rocksdb/_rocksdb.cpp: In function \u2018int __pyx_pf_7rocksdb_8_rocksdb_7Options_30max_grandparent_overlap_factor_2__set__(__pyx_obj_7rocksdb_8_rocksdb_Options*, PyObject*)\u2019:
rocksdb/_rocksdb.cpp:13536:23: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018max_grandparent_overlap_factor\u2019
   __pyx_v_self->opts->max_grandparent_overlap_factor = __pyx_t_1;
                       ^
rocksdb/_rocksdb.cpp: In function \u2018PyObject* __pyx_pf_7rocksdb_8_rocksdb_7Options_14filter_deletes___get__(__pyx_obj_7rocksdb_8_rocksdb_Options*)\u2019:
rocksdb/_rocksdb.cpp:18153:57: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018filter_deletes\u2019
   __pyx_t_1 = __Pyx_PyBool_FromLong(__pyx_v_self->opts->filter_deletes); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 1142; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                                         ^
rocksdb/_rocksdb.cpp:447:36: note: in definition of macro \u2018__Pyx_PyBool_FromLong\u2019
 #define __Pyx_PyBool_FromLong(b) ((b) ? (Py_INCREF(Py_True), Py_True) : (Py_INCREF(Py_False), Py_False))
                                    ^
rocksdb/_rocksdb.cpp: In function \u2018int __pyx_pf_7rocksdb_8_rocksdb_7Options_14filter_deletes_2__set__(__pyx_obj_7rocksdb_8_rocksdb_Options*, PyObject*)\u2019:
rocksdb/_rocksdb.cpp:18216:23: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018filter_deletes\u2019
   __pyx_v_self->opts->filter_deletes = __pyx_t_1;
                       ^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Complete output from command /nas1/buvana/rocksdb/rocksdb/pyrocks_test/bin/python -c "import setuptools, tokenize;__file__='/nas1/buvana/rocksdb/rocksdb/pyrocks_test/build/pyrocksdb/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-Lxk83M-record/install-record.txt --single-version-externally-managed --compile --install-headers /nas1/buvana/rocksdb/rocksdb/pyrocks_test/include/site/python2.7:
running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/init.py -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/interfaces.py -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/errors.py -> build/lib.linux-x86_64-2.7/rocksdb

creating build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/init.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/test_options.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

copying rocksdb/tests/test_db.py -> build/lib.linux-x86_64-2.7/rocksdb/tests

running egg_info

writing dependency_links to pyrocksdb.egg-info/dependency_links.txt

writing requirements to pyrocksdb.egg-info/requires.txt

writing pyrocksdb.egg-info/PKG-INFO

writing top-level names to pyrocksdb.egg-info/top_level.txt

warning: manifest_maker: standard file '-c' not found

reading manifest file 'pyrocksdb.egg-info/SOURCES.txt'

reading manifest template 'MANIFEST.in'

writing manifest file 'pyrocksdb.egg-info/SOURCES.txt'

copying rocksdb/_rocksdb.cpp -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/_rocksdb.pyx -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/backup.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/cache.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/comparator.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/db.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/env.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/filter_policy.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/iterator.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/logger.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/memtablerep.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/merge_operator.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/options.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/slice_.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/slice_transform.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/snapshot.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/status.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/std_memory.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/table_factory.pxd -> build/lib.linux-x86_64-2.7/rocksdb

copying rocksdb/universal_compaction.pxd -> build/lib.linux-x86_64-2.7/rocksdb

creating build/lib.linux-x86_64-2.7/rocksdb/cpp

copying rocksdb/cpp/comparator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp

copying rocksdb/cpp/filter_policy_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp

copying rocksdb/cpp/memtable_factories.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp

copying rocksdb/cpp/merge_operator_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp

copying rocksdb/cpp/slice_transform_wrapper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp

copying rocksdb/cpp/utils.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp

copying rocksdb/cpp/write_batch_iter_helper.hpp -> build/lib.linux-x86_64-2.7/rocksdb/cpp

running build_ext

building 'rocksdb._rocksdb' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/rocksdb

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c rocksdb/_rocksdb.cpp -o build/temp.linux-x86_64-2.7/rocksdb/_rocksdb.o -std=c++11 -O3 -Wall -Wextra -Wconversion -fno-strict-aliasing

cc1plus: warning: command line option \u2018-Wstrict-prototypes\u2019 is valid for C/ObjC but not for C++ [enabled by default]

rocksdb/_rocksdb.cpp: In function \u2018PyObject* __pyx_pf_7rocksdb_8_rocksdb_19PyBloomFilterPolicy_4create_filter(pyx_obj_7rocksdb_8_rocksdb_PyBloomFilterPolicy, PyObject)\u2019:

rocksdb/_rocksdb.cpp:5639:88: warning: conversion to \u2018int\u2019 from \u2018std::vectorrocksdb::Slice::size_type {aka long unsigned int}\u2019 may alter its value [-Wconversion]

 __pyx_t_6->CreateFilter(py_rocks::vector_data(__pyx_v_c_keys), __pyx_v_c_keys.size(), (&__pyx_v_dst));

                                                                                    ^

rocksdb/_rocksdb.cpp: In function \u2018PyObject* pyx_pf_7rocksdb_8_rocksdb_7Options_26expanded_compaction_factor___get(__pyx_obj_7rocksdb_8_rocksdb_Options*)\u2019:

rocksdb/_rocksdb.cpp:13229:56: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018expanded_compaction_factor\u2019

pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->opts->expanded_compaction_factor); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 873; __pyx_clineno = __LINE; goto __pyx_L1_error;}

                                                    ^

rocksdb/_rocksdb.cpp: In function \u2018int pyx_pf_7rocksdb_8_rocksdb_7Options_26expanded_compaction_factor_2__set(pyx_obj_7rocksdb_8_rocksdb_Options, PyObject)\u2019:

rocksdb/_rocksdb.cpp:13292:23: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018expanded_compaction_factor\u2019

__pyx_v_self->opts->expanded_compaction_factor = __pyx_t_1;

                   ^

rocksdb/_rocksdb.cpp: In function \u2018PyObject* pyx_pf_7rocksdb_8_rocksdb_7Options_24source_compaction_factor___get(__pyx_obj_7rocksdb_8_rocksdb_Options*)\u2019:

rocksdb/_rocksdb.cpp:13351:56: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018source_compaction_factor\u2019

pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->opts->source_compaction_factor); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 879; __pyx_clineno = __LINE; goto __pyx_L1_error;}

                                                    ^

rocksdb/_rocksdb.cpp: In function \u2018int pyx_pf_7rocksdb_8_rocksdb_7Options_24source_compaction_factor_2__set(pyx_obj_7rocksdb_8_rocksdb_Options, PyObject)\u2019:

rocksdb/_rocksdb.cpp:13414:23: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018source_compaction_factor\u2019

__pyx_v_self->opts->source_compaction_factor = __pyx_t_1;

                   ^

rocksdb/_rocksdb.cpp: In function \u2018PyObject* pyx_pf_7rocksdb_8_rocksdb_7Options_30max_grandparent_overlap_factor___get(__pyx_obj_7rocksdb_8_rocksdb_Options*)\u2019:

rocksdb/_rocksdb.cpp:13473:56: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018max_grandparent_overlap_factor\u2019

pyx_t_1 = __Pyx_PyInt_From_int(__pyx_v_self->opts->max_grandparent_overlap_factor); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 885; __pyx_clineno = __LINE; goto __pyx_L1_error;}

                                                    ^

rocksdb/_rocksdb.cpp: In function \u2018int pyx_pf_7rocksdb_8_rocksdb_7Options_30max_grandparent_overlap_factor_2__set(pyx_obj_7rocksdb_8_rocksdb_Options, PyObject)\u2019:

rocksdb/_rocksdb.cpp:13536:23: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018max_grandparent_overlap_factor\u2019

__pyx_v_self->opts->max_grandparent_overlap_factor = __pyx_t_1;

                   ^

rocksdb/_rocksdb.cpp: In function \u2018PyObject* pyx_pf_7rocksdb_8_rocksdb_7Options_14filter_deletes___get(__pyx_obj_7rocksdb_8_rocksdb_Options*)\u2019:

rocksdb/_rocksdb.cpp:18153:57: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018filter_deletes\u2019

pyx_t_1 = __Pyx_PyBool_FromLong(__pyx_v_self->opts->filter_deletes); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 1142; __pyx_clineno = __LINE; goto __pyx_L1_error;}

                                                     ^

rocksdb/_rocksdb.cpp:447:36: note: in definition of macro \u2018__Pyx_PyBool_FromLong\u2019

#define __Pyx_PyBool_FromLong(b) ((b) ? (Py_INCREF(Py_True), Py_True) : (Py_INCREF(Py_False), Py_False))

                                ^

rocksdb/_rocksdb.cpp: In function \u2018int pyx_pf_7rocksdb_8_rocksdb_7Options_14filter_deletes_2__set(pyx_obj_7rocksdb_8_rocksdb_Options, PyObject)\u2019:

rocksdb/_rocksdb.cpp:18216:23: error: \u2018struct rocksdb::Options\u2019 has no member named \u2018filter_deletes\u2019

__pyx_v_self->opts->filter_deletes = __pyx_t_1;

                   ^

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Cleaning up... Command /nas1/buvana/rocksdb/rocksdb/pyrocks_test/bin/python -c "import setuptools, tokenize;file='/nas1/buvana/rocksdb/rocksdb/pyrocks_test/build/pyrocksdb/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-Lxk83M-record/install-record.txt --single-version-externally-managed --compile --install-headers /nas1/buvana/rocksdb/rocksdb/pyrocks_test/include/site/python2.7 failed with error code 1 in /nas1/buvana/rocksdb/rocksdb/pyrocks_test/build/pyrocksdb Traceback (most recent call last): File "/nas1/buvana/rocksdb/rocksdb/pyrocks_test/bin/pip", line 11, in sys.exit(main()) File "/nas1/buvana/rocksdb/rocksdb/pyrocks_test/local/lib/python2.7/site-packages/pip/init.py", line 185, in main return command.main(cmd_args) File "/nas1/buvana/rocksdb/rocksdb/pyrocks_test/local/lib/python2.7/site-packages/pip/basecommand.py", line 161, in main text = '\n'.join(complete_log) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 42: ordinal not in range(128) (pyrocks_test)buvana@datamon3:~/rocksdb/rocksdb/pyrocks_test$

abuvaneswari avatar Oct 10 '16 16:10 abuvaneswari

it seems pyrocksdb hasn't been updated in a long time, for now my workaround is to install a version of rocksdb compatible with the python bindings... git clone https://github.com/facebook/rocksdb.git git reset --hard 8d7926a766f2ab9bd6e7aa8cba80b5d3ff26c52b

guybedo avatar Oct 26 '16 16:10 guybedo

please refer to python-rocksdb (https://github.com/twmht/python-rocksdb) if you have questions.

twmht avatar Apr 07 '17 16:04 twmht